Python AH Buyer / Seller / ffxiah.com scrubber

Applications/Tools written to make running DarkStar easier for the rest of us.
Fearmonger
Posts: 3
Joined: Wed Dec 26, 2018 9:34 pm

Re: Python AH Buyer / Seller / ffxiah.com scrubber

Post by Fearmonger » Fri Dec 28, 2018 6:16 pm

There are (2) column options "Rate01, Rate12". Can anyone explain what this does and give me an example of how it would be written please?

User avatar
TeoTwawki
Developer
Posts: 523
Joined: Mon Jul 15, 2013 9:50 pm

Re: Python AH Buyer / Seller / ffxiah.com scrubber

Post by TeoTwawki » Tue Jan 29, 2019 7:53 pm

y'all need to realize you'd get better results posting issues and requests to his tracker. It is highly unlikely that the author would come running to the DSP forums with every post to see what was said. https://github.com/AdamGagorik/pydarkstar/issues
Hi, I run The Demiurge server.

Code: Select all

PLS USE [code]CODE TAGS[/code] WHEN POSTING CODE
Image
Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live. - Martin Golding
DO NOT PRIVATE MESSAGE ME ABOUT BUGS

User avatar
TeoTwawki
Developer
Posts: 523
Joined: Mon Jul 15, 2013 9:50 pm

Re: Python AH Buyer / Seller / ffxiah.com scrubber

Post by TeoTwawki » Tue Jan 29, 2019 7:54 pm

Fearmonger wrote:
Fri Dec 28, 2018 6:16 pm
There are (2) column options "Rate01, Rate12". Can anyone explain what this does and give me an example of how it would be written please?
They aren't used (yet). You can see the intended future use right on the instruction page. http://adamgagorik.github.io/pydarkstar ... usage.html
Hi, I run The Demiurge server.

Code: Select all

PLS USE [code]CODE TAGS[/code] WHEN POSTING CODE
Image
Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live. - Martin Golding
DO NOT PRIVATE MESSAGE ME ABOUT BUGS

User avatar
evenmonkeys
Posts: 78
Joined: Thu Feb 13, 2014 1:53 am
Location: Midwest, US

Re: Python AH Buyer / Seller / ffxiah.com scrubber

Post by evenmonkeys » Sun Feb 10, 2019 3:26 pm

I am having an issue when I am trying to run the scrub command to pull new prices for a fresh install. Could anyone maybe give some ideas where to be looking for the issue? I deleted attempts 2-10 as they all said the same thing.

Code: Select all

[2019-02-10 14:23:49,594][ 3996][INFO ]: (0x29b5588) Options: config     = config.yaml
[2019-02-10 14:23:49,594][ 3996][INFO ]: (0x29b5588) Options: verbose    = True
[2019-02-10 14:23:49,595][ 3996][INFO ]: (0x29b5588) Options: silent     = False
[2019-02-10 14:23:49,595][ 3996][INFO ]: (0x29b5588) Options: stub       = items
[2019-02-10 14:23:49,595][ 3996][INFO ]: (0x29b5588) Options: overwrite  = False
[2019-02-10 14:23:49,595][ 3996][INFO ]: (0x29b5588) Options: backup     = True
[2019-02-10 14:23:49,596][ 3996][INFO ]: (0x29b5588) Options: stock01    = 3
[2019-02-10 14:23:49,596][ 3996][INFO ]: (0x29b5588) Options: stock12    = 3
[2019-02-10 14:23:49,596][ 3996][INFO ]: (0x29b5588) Options: itemids    = ????
[2019-02-10 14:23:49,596][ 3996][INFO ]: (0x29b5588) Options: urls       = ????
[2019-02-10 14:23:49,597][ 3996][DEBUG]: (0x3d940f0) FFXIAHScrubber: init
[2019-02-10 14:23:49,598][ 3996][DEBUG]: (0x3d940f0) FFXIAHScrubber: forcing redownload of data
[2019-02-10 14:23:49,598][ 3996][DEBUG]: (0x3d940f0) FFXIAHScrubber: getting category urls
[2019-02-10 14:23:49,598][ 3996][DEBUG]: (0x3d940f0) FFXIAHScrubber: open https://www.ffxiah.com/browse
[2019-02-10 14:23:49,600][ 3996][ERROR]: urlopen failed (attempt 1)
Traceback (most recent call last):
  File "C:\Users\Highwind\Desktop\ah\pydarkstar\scrubbing\scrubber.py", line 33, in soup
    handle = urlopen(url)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 525, in open
    response = self._open(req, data)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 548, in _open
    'unknown_open', req)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1387, in unknown_open
    raise URLError('unknown url type: %s' % type)
urllib.error.URLError: <urlopen error unknown url type: https>
[2019-02-10 14:24:01,679][ 3996][ERROR]: the maximum urlopen attempts have been reached
[2019-02-10 14:24:01,681][ 3996][ERROR]: caught unhandled excetion
Traceback (most recent call last):
  File "C:\Users\Highwind\Desktop\ah\pydarkstar\logutils.py", line 45, in capture
    yield
  File "C:\Users\Highwind\Desktop\ah\bin\scrub.py", line 21, in <module>
    pydarkstar.apps.scrub.run.main()
  File "C:\Users\Highwind\Desktop\ah\pydarkstar\apps\scrub\run.py", line 35, in main
    data = scrubber.scrub(force=True, threads=-1, urls=opts.urls, ids=opts.itemids)
  File "C:\Users\Highwind\Desktop\ah\pydarkstar\scrubbing\ffxiah.py", line 63, in scrub
    urls = self._get_category_urls()
  File "C:\Users\Highwind\Desktop\ah\pydarkstar\scrubbing\ffxiah.py", line 194, in _get_category_urls
    soup = self.soup(path)
  File "C:\Users\Highwind\Desktop\ah\pydarkstar\scrubbing\scrubber.py", line 33, in soup
    handle = urlopen(url)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 525, in open
    response = self._open(req, data)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 548, in _open
    'unknown_open', req)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1387, in unknown_open
    raise URLError('unknown url type: %s' % type)
urllib.error.URLError: <urlopen error unknown url type: https>
[2019-02-10 14:24:01,704][ 3996][INFO ]: exit

Post Reply