[Bug] Image Scrapers

[dangerousmind]

When I pasted the url to the Term / URL box with

Code: Select all

http://www.tumblr.com/search/watch+dogs

The scraper doesn't really find the right images on that page. Here's the log.

Code: Select all

Pull src: http://www.tumblr.com/search/watch+dogs
Do page: http://www.tumblr.com/search/watch+dogs
Found: 7 images on : http://www.tumblr.com/search/watch+dogs
Looping: 7images with: 4 threads
Download: http://24.media.tumblr.com/avatar_9ad0c206317f_64.png
Download: http://38.media.tumblr.com/avatar_ace1e6137fa9_64.png
Download: http://37.media.tumblr.com/avatar_414652617446_64.png
Download: http://38.media.tumblr.com/avatar_de4924f8de34_64.png
Scraped: http://38.media.tumblr.com/avatar_de4924f8de34_64.png
* IGNORE: Width less than minimum: 100
Processed: 1 images
Download: http://38.media.tumblr.com/avatar_c0b5a42c0dac_64.png
Scraped: http://37.media.tumblr.com/avatar_414652617446_64.png
* IGNORE: Width less than minimum: 100
Processed: 2 images
Download: http://31.media.tumblr.com/avatar_d4a50ae2feca_64.png
Scraped: http://38.media.tumblr.com/avatar_c0b5a42c0dac_64.png
* IGNORE: Width less than minimum: 100
Processed: 3 images
Download: http://37.media.tumblr.com/tumblr_n78vd15hOw1s8hg7to1_1402893254_cover.jpg
Scraped: http://38.media.tumblr.com/avatar_ace1e6137fa9_64.png
* IGNORE: Width less than minimum: 100
Processed: 4 images
Scraped: http://31.media.tumblr.com/avatar_d4a50ae2feca_64.png
* IGNORE: Width less than minimum: 100
Processed: 5 images
Scraped: http://37.media.tumblr.com/tumblr_n78vd15hOw1s8hg7to1_1402893254_cover.jpg
* SAVED: http://37.media.tumblr.com/tumblr_n78vd15hOw1s8hg7to1_1402893254_cover.jpg to C:\Users\Randize\Desktop\Cinematic\tumblr_n78vd15hOw1s8hg7to1_1402893254_cover.jpg
Processed: 6 images
Scraped: http://24.media.tumblr.com/avatar_9ad0c206317f_64.png
* IGNORE: Width less than minimum: 100
Processed: 7 images
Got pages: -1
Do page: http://www.tumblr.com/search/watch+dogs/page/2
No posts found on page: 2 - Assumed page doesn't exist so stopping

And when I do a tagged search

Code: Select all

http://www.tumblr.com/tagged/watch-dogs

This log says:

Code: Select all

Pull src: http://www.tumblr.com/tagged/watch-dogs
Do page: http://www.tumblr.com/tagged/watch-dogs
* ERROR: Exception of type '䆨쉔샧朔ꣿ饟지' was thrown.

[martin@rootjazz]

I Think you are entering the term wrong. Let me load up and test. Been a while since I used this module so cannot quite remember how it works. I prefer to use the RSS feeds.

Give me a few minutes and I'll get back to you

[martin@rootjazz]

so ignore me above. your term is correct. Something appears to have changed that broke the functionality.

Will need a few more minutes....

[martin@rootjazz]

Tagged search seems to have been updated to the new style of search. The code is for the old search. For now, I am going to have to remove this functionality and make a note to get image scraper via search re-added should the demand be there.

[dangerousmind]

Oh man.. hope there are more people demand for this functionality, I love the precise search function for reblogs. That module works perfectly.

For now I just have to scrape images using imacros then.

[martin@rootjazz]

hmmm strangeness.

If I do a normal search 'watch dogs'
I get URL
http://www.tumblr.com/search/WATCH+DOGS

Then click a tag
#watch dogs
go to
http://www.tumblr.com/search/%23WATCH+DOGS
which is
http://www.tumblr.com/search/#WATCH+DOGS

so search hashtag.

If I go to old tagged URL
http://www.tumblr.com/tagged/watch-dogs

then this works as well.

so am not entirely sure what is going on with the search.

Either way the scrape search and download isn't working. It needs updating for all search including tagged search whether that is hashtag or the old tagged URL.

[dangerousmind]

Yea the entered term for other functionality is parsing the right url and they pull of the correct page as I checked.

[martin@rootjazz]

Just in case my random mutterings to myself didn't make too much sense (they barely do to me tbh)

Scrape images from search URL is broken.

To be updated with full search capability.

In all honest, will probably be early next week unless there is a sudden great demand to do it know. My week is pretty much planned to do all the things I promised last week and ran out of time for.