Why does it always saved custom replies in INBOX SCRAPE even though I unchecked "custom replies"?


eddielp123
Posts: 447
Joined: Tue Feb 11, 2020 2:53 pm

Re: Why does it always saved custom replies in INBOX SCRAPE even though I unchecked "custom replies"?

Post by eddielp123 » Tue Jul 14, 2020 11:06 am

Okay so the problem is that right now, I have 1000 accounts and everytime I scrape messages it takes up to 3 hours.
Even when I delete all messages before scraping and then only scrape new messages, at a certain point it will take like 7+ minutes for an inbox scrape action to perform, whie at the start of the 1000 accs msg scrape its usually pretty quick (1 minute max). But as said, the more time passes the slower it gets. This is from my 12 GB ram Laptop btw

I don't know what can be done to speed this process up. Let me know if i could submit logs to help out

User avatar
martin@rootjazz
Site Admin
Posts: 26010
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Why does it always saved custom replies in INBOX SCRAPE even though I unchecked "custom replies"?

Post by martin@rootjazz » Tue Jul 14, 2020 1:22 pm

eddielp123 wrote:
Tue Jul 14, 2020 11:06 am
Let me know if i could submit logs to help out
Well if you don't submit logs there is nothing I can do as I cannot see the issue. Think of it like calling a mechanic and telling them you have a problem with the engine. They have to see the car, if they don't, they cannot help you.

Same with logs, always submit.


And as always:
For future reference, please, read through this post and submit your issue accordingly. Not everything will be relevant to your issue, but have a read as it gives an idea of the information you can provide to help me help you as quickly as possible without delays in having to request additional information :)

https://rootjazz.com/forum/viewtopic.php?f=23&t=1634


If you can generate issue specific logs all the better:



Please perform the following:

Open the program. Delete or pause all existing actions. If actions are already running due to autorun on startup, disable autorun at startup and restart the app. All actions must be waiting, then pause all (or delete)


Delete existing logs
HELP > LOGS > VIEW
delete all files in the folder that opens


Perform the action:
***

Run an inbox scrape with a single account to complete

***

PLEASE NOTE: Do not perform other actions at the same time. Only use one thread. Do not run any actions after this action.


Then submit logs:
HELP > LOGS > SUBMIT

and let me know the LOGS ID (just the number part is sufficient)




Just to repeat as people tend to ignore this

PLEASE NOTE: Do not perform other actions at the same time. Only use one thread. Do not run any actions after this action. Don't do anything, run the action, wait for it to finish, then submit logs

Regards,
Martin

eddielp123
Posts: 447
Joined: Tue Feb 11, 2020 2:53 pm

Re: Why does it always saved custom replies in INBOX SCRAPE even though I unchecked "custom replies"?

Post by eddielp123 » Tue Jul 14, 2020 6:44 pm

martin@rootjazz wrote:
Tue Jul 14, 2020 1:22 pm
eddielp123 wrote:
Tue Jul 14, 2020 11:06 am
Let me know if i could submit logs to help out
Well if you don't submit logs there is nothing I can do as I cannot see the issue. Think of it like calling a mechanic and telling them you have a problem with the engine. They have to see the car, if they don't, they cannot help you.
Submitting logs just seems kinda useless especially for single accounts since the issue only occurs when too many accounts scrape messages, as identified:
martin@rootjazz wrote:
Mon Jun 29, 2020 4:40 pm
Delay is due to loading in received message to the control, you have lots of messages in the db so it is taking some time. If you don't want the existing messages, you can delete any unwanted from teh control, by selecting the rows (multi select with shift / control as normal per windows) and press DELETE key on keyboard.

I'll take a look to see if can be speeded up
Well so the problem is basically this, when scraping messages with 1000 accounts, even if I delete them off my db before I scrape, by the time the first 500 accounts messages are scraped there’s so many again in the DB that the other 500 will scrape just really really slow. I don’t know if there’s anything that can be done about it, except for maybe scraping messages for 200 accounts, then deleting, then the next 200 and so on.

User avatar
martin@rootjazz
Site Admin
Posts: 26010
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Why does it always saved custom replies in INBOX SCRAPE even though I unchecked "custom replies"?

Post by martin@rootjazz » Wed Jul 15, 2020 1:54 pm

eddielp123 wrote:
Tue Jul 14, 2020 6:44 pm


Submitting logs just seems kinda useless especially for single accounts since the issue only occurs when too many accounts scrape messages, as identified:
martin@rootjazz wrote:
Mon Jun 29, 2020 4:40 pm
Delay is due to loading in received message to the control, you have lots of messages in the db so it is taking some time. If you don't want the existing messages, you can delete any unwanted from teh control, by selecting the rows (multi select with shift / control as normal per windows) and press DELETE key on keyboard.

I'll take a look to see if can be speeded up
Well so the problem is basically this, when scraping messages with 1000 accounts, even if I delete them off my db before I scrape, by the time the first 500 accounts messages are scraped there’s so many again in the DB that the other 500 will scrape just really really slow. I don’t know if there’s anything that can be done about it, except for maybe scraping messages for 200 accounts, then deleting, then the next 200 and so on.
As I said, I can look at efficiencies, but it will require you to provide what has been asked for. If you don't want to, or don't feel anything can be done, or just feel you know best :p, fair enough.

but if you want me to look at it, you will have to just accept that I have an idea of what I am doing and indulge me my requests for information :)

eddielp123
Posts: 447
Joined: Tue Feb 11, 2020 2:53 pm

Re: Why does it always saved custom replies in INBOX SCRAPE even though I unchecked "custom replies"?

Post by eddielp123 » Wed Jul 15, 2020 2:29 pm

martin@rootjazz wrote:
Wed Jul 15, 2020 1:54 pm
eddielp123 wrote:
Tue Jul 14, 2020 6:44 pm


Submitting logs just seems kinda useless especially for single accounts since the issue only occurs when too many accounts scrape messages, as identified:
martin@rootjazz wrote:
Mon Jun 29, 2020 4:40 pm
Delay is due to loading in received message to the control, you have lots of messages in the db so it is taking some time. If you don't want the existing messages, you can delete any unwanted from teh control, by selecting the rows (multi select with shift / control as normal per windows) and press DELETE key on keyboard.

I'll take a look to see if can be speeded up
Well so the problem is basically this, when scraping messages with 1000 accounts, even if I delete them off my db before I scrape, by the time the first 500 accounts messages are scraped there’s so many again in the DB that the other 500 will scrape just really really slow. I don’t know if there’s anything that can be done about it, except for maybe scraping messages for 200 accounts, then deleting, then the next 200 and so on.
As I said, I can look at efficiencies, but it will require you to provide what has been asked for. If you don't want to, or don't feel anything can be done, or just feel you know best :p, fair enough.

but if you want me to look at it, you will have to just accept that I have an idea of what I am doing and indulge me my requests for information :)
don't get me wrong, i always think that you got the best solution for these problems :D
but we already identified the problem ( I THINK ) that's why i quoted that post. but i'm going to submit logs again today, i didn't want to come off like mr. know it all :D

eddielp123
Posts: 447
Joined: Tue Feb 11, 2020 2:53 pm

Re: Why does it always saved custom replies in INBOX SCRAPE even though I unchecked "custom replies"?

Post by eddielp123 » Wed Jul 15, 2020 6:09 pm

Running a test run now with 1000 accounts to submit logs... will take a few hours but will post it here once ready :D

eddielp123
Posts: 447
Joined: Tue Feb 11, 2020 2:53 pm

Re: Why does it always saved custom replies in INBOX SCRAPE even though I unchecked "custom replies"?

Post by eddielp123 » Wed Jul 15, 2020 8:30 pm

Okay so I just ran a test with 1 THREAD maximum at a time, it ran pretty normal to my surprise, always about 1-2 minutes scrape time. However with 1000 accounts this isn't really an option since I want to answer all messages and I can't afford waiting 17 hours for all messages to be scraped :D So I always run it with at least 10-15 threads at the same time. Or could you suggest any other good method of scraping messages for 1000 accounts, for them to constantly be more or less up to date?


The log ID for this test is: 80974

I've also just ran a test for 10 maximum threads with 62 accounts and here the issue seems more apparent and present. At the beginning the actions will take about 0:30-2 minutes, but as time passes they start taking about 4+ minutes per action.
So the problem seems to be in the amount of threads that I am running at the same time (?) , but I don't really know what other option there is when wanting to scrape messages for 1000 accounts since I can't afford to wait 15+ hours for all messages to be scraped across all accounts..

I don't know if the log for this will even help because it will probably be all over the place for you since I'm running 10 threads at the same time?

Log ID for 10 maximum threads test (on my 12 GB RAM laptop btw): 91832


Of course I'd be thankful too for any suggestions, maybe out of your own experience, on how to best approach scraping messages for 1000 accounts?

User avatar
martin@rootjazz
Site Admin
Posts: 26010
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Why does it always saved custom replies in INBOX SCRAPE even though I unchecked "custom replies"?

Post by martin@rootjazz » Fri Jul 17, 2020 1:43 pm

eddielp123 wrote:
Wed Jul 15, 2020 8:30 pm
The log ID for this test is: 80974

Many thanks. I'll go through it. Am flat out with a critical issue with soundcloud program which is #1 priority now as it relates to accounts getting banned.

Once that is sorted, I will be clearing outsanding support

eddielp123
Posts: 447
Joined: Tue Feb 11, 2020 2:53 pm

Re: Why does it always saved custom replies in INBOX SCRAPE even though I unchecked "custom replies"?

Post by eddielp123 » Fri Jul 17, 2020 2:46 pm

martin@rootjazz wrote:
Fri Jul 17, 2020 1:43 pm
eddielp123 wrote:
Wed Jul 15, 2020 8:30 pm
The log ID for this test is: 80974

Many thanks. I'll go through it. Am flat out with a critical issue with soundcloud program which is #1 priority now as it relates to accounts getting banned.

Once that is sorted, I will be clearing outsanding support
Thanks a lot. Take your time! Please keep in mind I submitted two logs in the post before, in the second one right here the issue should be more apparent
eddielp123 wrote:
Wed Jul 15, 2020 8:30 pm
Log ID for 10 maximum threads test (on my 12 GB RAM laptop btw): 91832

Post Reply