REQUEST : NEW SCRAPE METHOD

Post Reply
redkooh
Posts: 22
Joined: Sat Jul 24, 2021 8:40 am

REQUEST : NEW SCRAPE METHOD

Post by redkooh »

hi martin,
i have a request to scrape feed url from millions of username database without need of login

each user, i need 1 url from their feed post

for example,
1. without login, i can go to instagram.com/instagram and pick 1 on the picture to copy the url
2. repeat with next username

with this method, we can do unlimited url scrape without having the risk of getting ban

note: rotating residential proxy is optional for this method in case the ip might getting ban.
User avatar
martin@rootjazz
Site Admin
Posts: 28152
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: REQUEST : NEW SCRAPE METHOD

Post by martin@rootjazz »

You are asking for a WEB SCRAPER module, not PHONE API. At this time the program is just a phone app automated tool, it doesn't automate the website.

I'll make a note to investigate.


--------------------------------

Ok had a quick look, should be ok to get the info fairly easily in a single call *IF* the proxy is able to view the profile page NOT LOGGED IN (not all can and will be redirected to the login page, but those that can get

profile details
first 20 medias
users tagged in medias (tagged not @mentioned)


Coding this won't be difficult BUT allowing you to use specific external proxies, that will be more complex as nothing exist for that currently.

Anyway, I'll take a look, remind me next week sometime



Regards,
Martin
redkooh
Posts: 22
Joined: Sat Jul 24, 2021 8:40 am

Re: REQUEST : NEW SCRAPE METHOD

Post by redkooh »

hi thanks! i will remind you next week
User avatar
martin@rootjazz
Site Admin
Posts: 28152
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: REQUEST : NEW SCRAPE METHOD

Post by martin@rootjazz »

redkooh wrote: Fri Oct 08, 2021 5:28 pm hi thanks! i will remind you next week
But not monday - monday is catch up from weekend. So tuesday ./ weds best :-)
redkooh
Posts: 22
Joined: Sat Jul 24, 2021 8:40 am

Re: REQUEST : NEW SCRAPE METHOD

Post by redkooh »

Hi! its almost Tuesday and i guess you are good to go hehe

I really hope you can create this as you said it is simple enough.
thank you so much martin really appreciate your good work :)
User avatar
martin@rootjazz
Site Admin
Posts: 28152
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: REQUEST : NEW SCRAPE METHOD

Post by martin@rootjazz »

redkooh wrote: Mon Oct 11, 2021 2:13 pm Hi! its almost Tuesday and i guess you are good to go hehe

I really hope you can create this as you said it is simple enough.
thank you so much martin really appreciate your good work :)
Won't be tuesday, perhaps wednesday - most likely thursday. hopefully ready for release Friday
redkooh
Posts: 22
Joined: Sat Jul 24, 2021 8:40 am

Re: REQUEST : NEW SCRAPE METHOD

Post by redkooh »

HI MARTIN!

Good news! its Thursday 😁
User avatar
martin@rootjazz
Site Admin
Posts: 28152
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: REQUEST : NEW SCRAPE METHOD

Post by martin@rootjazz »

redkooh wrote: Thu Oct 14, 2021 5:43 am HI MARTIN!

Good news! its Thursday 😁
Sorry had a really bad week, power cuts, internet down, family issues. Will have to bump it to next week, sorry
redkooh
Posts: 22
Joined: Sat Jul 24, 2021 8:40 am

Re: REQUEST : NEW SCRAPE METHOD

Post by redkooh »

Sure take your time man👋
Post Reply