1.9.6.4 Scrape notes of post from list empty files

[snibi]

Hello,

I have just tried to scrape notes of post from list of /post/ urls in TJ 1.9.6.4

it is producing empty files here.

To reproduce:
Scrape tab
enter url of post file
custom search users -> scrape notes of posts
hit search

[martin@rootjazz]

testing...

[martin@rootjazz]

Working fine here.

Will need more info to debug further

Please, read through this post and submit your issue accordingly. Not everything will be relevant to your issue, but have a read as it gives an idea of the information you can provide to help me help you as quickly as possible without delays in having to request additional information

https://rootjazz.com/forum/viewtopic.php?f=23&t=1634



Regards,
Martin

[snibi]

What program and version are you using do check if there is an update:
TJ 1964, software is up to date.

Describe what is going wrong:
it scrapes, create files but writes 0 byte files.

Describe what was expected:
it scrapes, create files and write files full of urls.

Describe the steps to replicate your issue. A list of instruction steps, include all inputs / options you set (screenshots can be useful here)
Scrape tab
enter url of posts file
custom search users -> scrape notes of posts
hit search

If you are setting any specific options for the action, detail those.
none

Include the processing logs, that show your issue (processing logs are shown when you double click the action in the processing tab
Logs ID 94488
Logs ID 38416

[martin@rootjazz]

What program and version are you using do check if there is an update:
TJ 1964, software is up to date.

Describe what is going wrong:
it scrapes, create files but writes 0 byte files.

Describe what was expected:
it scrapes, create files and write files full of urls.

Describe the steps to replicate your issue. A list of instruction steps, include all inputs / options you set (screenshots can be useful here)
Scrape tab
enter url of posts file
custom search users -> scrape notes of posts
hit search

If you are setting any specific options for the action, detail those.
none

Include the processing logs, that show your issue (processing logs are shown when you double click the action in the processing tab
Logs ID 94488
Logs ID 38416

strange, exactly the same as me then. Was expecting something different which would explain it. The post did have notes I assume?

checking logs...

[martin@rootjazz]

22:23:28: * Cancelled:

one action shows it was cancelled? Was that you? If so, don't cancel the action.

22:15:01: getWebResponse Proxy: xxx.xxx.xxx.104:8085 ConnectFailure
Exception: WebException
Unable to connect to the remote server

other action is a big scrape, seems like you got to 31866 results, but tumblr started to block you which caused the action to fail. This flow then failed to save

2:15:20: Scraped: 32617 from: https://url
22:15:20: Total scraped: 32617
22:15:20: Results of search: Scrape Notes Of Post with: https://url
22:15:24: Paused for 4secs
22:15:24: Handle results: 0 nextstep: 2/1
22:15:24: End of chain: Store results: 0
22:15:24: Finished search: results

I assume if you just try and scrape 100 results, it will all work correctly. Can you confirm this, just so I know the issue for you is ONLY when something goes wrong and what path the code takes results in no results saved for some reason

[snibi]

Yes with Logs ID 94488, i had big scrape, and was waiting until the first file was written with 0 then cancelled and
set to max 99 results with Logs ID 38416

Here is another one Logs ID 72346

[martin@rootjazz]

so setting 99 it STILL doesn't work for you. It is nothing to do with the failing and dropping out then.

This is strange as I said, for me, it worked first time.

Let me try to map my logs to your logs to find out where exactly it goes wrong. I may need to get you a debugging build out to try to find out why it works for me and not for you



Regards,
Martin

[martin@rootjazz]

Please re-run a small scrape test of 100 results. If fails again (probably it will) submit logs
HELP > LOGS > SUBMIT

then send your logs ID - the first 4 numbers is sufficient (displayed after successful uploading of logs)

[snibi]

Thank you for the new build, i have tested and it seems it has worked now, no more 0 bytes files.

ID: 5034