Scrape Posts of Blog, next Scrape Notes of Posts

[snibi]

Hi all,

With the new scrape module i am confused.

The procedure on the old one where we put just a URL, or a filepath was this
1. Scrape Posts of Blog
2. Scrape Notes of Posts

WITHOUT any account.

Now at the new scraper module i cant even find "scrape all posts from blog".
And it wants an account - which will maybe banned after just some scrapes?

How do you do it?

[martin@rootjazz]

I will get a tutorial out this week


To tidy up the scrape tab, everything is now handled by the custom search builder as it can perform all searches and more.

As for not requiring an account, I could remove that requirement, but it means relying on you the user not to setup a scrape without an account that requires the account.... next update will include this

But basically, just open the custom search builder, select the scrape you want to perform

and that's it

[snibi]

Thanks Martin!

As for not requiring an account, I could remove that requirement, but it means relying on you the user not to setup a scrape without an account that requires the account.... next update will include this

I think its better to remove that requirement, thanks! Those who already know that they need an account will put an account.

For the search, "1. Scrape Posts of Blog"
I suppose i do it right, but i am missing a "all posts from blog" here:

[martin@rootjazz]

Thanks Martin!

As for not requiring an account, I could remove that requirement, but it means relying on you the user not to setup a scrape without an account that requires the account.... next update will include this

I think its better to remove that requirement, thanks!

lol, of course you do, it suits you. Until I can 10s of complaints that search isn't returning more than 10 results.....

Those who already know that they need an account will put an account

and those that don't?


Anyway

[snibi]

that search isn't returning more than 10 results.....

Now i understand, thanks for clearing that up, that completely changed my mind.

and those that don't?

i will use from now on accounts for scraping

Now i only need to know how to scrape all blogposts, what to click.

[martin@rootjazz]

https://rootjazz.com/tumblingjazz/updatetesting.html

[snibi]

Thank you Martin! What do i select now for ALL blogposts of a blogurl in "Custom Search Posts"?

[martin@rootjazz]

Thank you Martin! What do i select now for ALL blogposts of a blogurl in "Custom Search Posts"?

Recent Posts Of

It will then scrape back from the most recent all the way back until the requested number are scraped.


In the custom search module, you can double click the search steps to specify the number of items for each step you want, just helps the program in some searches to know what you want

[snibi]

I understand! Now i had selected an account, it tells me that i have not selected an account. But the scrape did work!

Next i feeded the "Custom Search Users" with the scraped_list_of_posts.txt and hit Search.

Foreach line it presents me a new Info "Added to processor", multiple times i needed to click Ok, but after i hit 532x Ok,
it was all added to processor and scraping! Thanks again!

[martin@rootjazz]

Foreach line it presents me a new Info "Added to processor", multiple times i needed to click Ok, but after i hit 532x Ok,
it was all added to processor and scraping! Thanks again!

Fix one problem, find another *sigh*

It should only say that once, I Will fix and update today



Regards,
Martin