Tried again, restored my backups of the global logs, and created 4 jobs this time, each shared across approx 50 accounts. The same problem, within 5 minutes I noticed the global logs had been shrunk to a few KB.
Tried running a single job across 810 accts overnight, just enough to send 1 DM from each account, but it's been running for 11 hrs now so there's no way this is scalable. But, log files are still intact.
Duplicate DM Actions
Re: Duplicate DM Actions
Testing more, running smaller jobs again, 1 acct per job sending between 3 and 15 DM's. I queued up about 50 jobs, but only let the processor run 3 threads.
Running 3 jobs in the processer logs seemed to be surviving. Soon after adding a 4th thread one of the large global logs was shrunk, 3.5mb down to 2.27kb.
I paused the processor and checked the raw logs for the 4 active threads. Nothing there seems strange, they all finished scraping, then checked the results against processed logs and started sending DM's.
Running 3 jobs in the processer logs seemed to be surviving. Soon after adding a 4th thread one of the large global logs was shrunk, 3.5mb down to 2.27kb.
I paused the processor and checked the raw logs for the 4 active threads. Nothing there seems strange, they all finished scraping, then checked the results against processed logs and started sending DM's.
Re: Duplicate DM Actions
Tested overnight with just a single processor thread. Works absolutely fine, all global logs steadily increasing in size. However, it's far too slow to possibly send DM's across all accounts.
@Martin, as far as I can tell some kind of lock occurs when multiple threads try to update the logs simultaneously which results in them being destroyed.
@Martin, as far as I can tell some kind of lock occurs when multiple threads try to update the logs simultaneously which results in them being destroyed.
Re: Duplicate DM Actions
One more update for now. To check if I had some kind of corrupted installation or data I tested this on another instance of TwitterDub I have that runs only about 70 accounts in a different niche. Exactly the same issue when I ran DM's across all accounts in parallel, existing global logs were shrunk/deleted during the process.
- martin@rootjazz
- Site Admin
- Posts: 34706
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Duplicate DM Actions
ok found the thread from your email, looking into now
- martin@rootjazz
- Site Admin
- Posts: 34706
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Duplicate DM Actions
Alexiom wrote: ↑Sun May 28, 2023 5:58 pm Hi Martin, I've done a ton of testing and recorded everything. Sorry for the massive reply, but hope the info is helpful:
1) Deleted existing raw logs
2) Confirmed there are two existing global DM logs, the original, and the one that got added when I ran "Convert Shared to Global"
3) Checked size of both log files (DM_Logs = 78kb and auto = 49kb)
4) Had 5 accts send 5 DM's each: 5 custom search results, 5 max actions, unique items selected, 1 job per acct.
This ran much faster than expected, I confirmed that all filters were set as usual.
No actions were skipped, and no previously processed items were found.
5) Checked global log sizes: Now DM_Logs = 80kb and auto = 51kb
6) Added new global log to all accts (now 3 each)
7) Ran same DM action on same 5 accts.
8) Checked global log sizes, now DM_Logs = 82kb, auto = 53kb, DM_Logs2 = 3kb (all growing)
9) Checked how many previously processed jobs were loaded in all raw logs for both runs:
3832 -> 3992
4077 -> 4237
3959 -> 4119
3878 -> 4038
4116 -> 4264 (This acct had previously been returning 150k+ processed items)
These numbers all seem very small, I've sent many thousands of DM's across all accts, but much less than 4k on each acct.
10) Submitted logs: 31806
11) Since the number of previously processed items seems very low, I ran "Convert Shared to Global" again across my 810 active accounts and this time got a global log of 3.5mb!!! Now all accts have 4 global logs.
12) For a 3rd time, ran the same DM process sending 5 DM's from the same 5 accts.
This pulled in WAYYY more "previous db logs"
3832 -> 3992 -> 85353
4077 -> 4237 -> 85598
3959 -> 4119 -> 85480
3878 -> 4038 -> 85399
4116 -> 4264 -> 85353
All jobs are still completed without finding a duplicate processed item.
Unlike the first time I converted shared to global, the new global log did not shrink when I ran the jobs. Last time I got a similar size file, about 3.5 MB, then sent my DM's, and the global log had shrunk to a few KB, like all of the previously processed items were blown away.
I've been running 1 job per acct, so 800 + jobs, each job sends just a few DM's but repeats 5 times. I run 150+ threads in the processor. I'm wondering if so many parallel jobs all trying to update the same log file could corrupt it? Maybe I should use "Share Across Accounts" to run fewer parallel jobs?
13) Submitted logs again: 99119
11) Emailed you 15x raw logs and before and after snapshots of the 4 global logs.
Hope this is useful. Thanks again.
Something isn't right here, sorry I should have said before, but I will need to see your datebase and analyse it. Maybe there has been database corruption, or the functions haven't run correctly.
Can you submit your database please
TOOLS > CREATE DB BACKUP AND SEND TO SUPPORT
then send your DB ID - the first 4 numbers is sufficient (displayed after successful uploading of logs)
Regards,
Martin
- martin@rootjazz
- Site Admin
- Posts: 34706
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Duplicate DM Actions
ok..... if file corrupted, perhaps it was backed up and a new one created... I cannot think how it was wiped another way. AS above, I need your database for reviewAlexiom wrote: ↑Sun May 28, 2023 10:06 pm Hi Martin,
Quick update, things were running fine until I tried to run 20 parallel threads (1 acct per thread sending DM's). Within a few minutes something happened and shrunk the newest log file down to 22.7 KB (from 3541 KB).
So possibly there is an issue with parallel threads clashing...
- martin@rootjazz
- Site Admin
- Posts: 34706
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Duplicate DM Actions
No, it definitely sounds like a bug. let me check the new logs.Alexiom wrote: ↑Tue May 30, 2023 12:12 pm One more update for now. To check if I had some kind of corrupted installation or data I tested this on another instance of TwitterDub I have that runs only about 70 accounts in a different niche. Exactly the same issue when I ran DM's across all accounts in parallel, existing global logs were shrunk/deleted during the process.
But if you can confirm the actions.
5 processing threads
5 DM actions.
How many accounts in the action? Can you explain what they were doing exactly, so I can try and recreate. If it happens all the time within 5 minutes I should be able to recreate fairly easily... I hope
- martin@rootjazz
- Site Admin
- Posts: 34706
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Duplicate DM Actions
if could send the database from this instance, it will probably be easier to see / confirm what happeningAlexiom wrote: ↑Tue May 30, 2023 12:12 pm One more update for now. To check if I had some kind of corrupted installation or data I tested this on another instance of TwitterDub I have that runs only about 70 accounts in a different niche. Exactly the same issue when I ran DM's across all accounts in parallel, existing global logs were shrunk/deleted during the process.
Can you submit your database please
TOOLS > CREATE DB BACKUP AND SEND TO SUPPORT
then send your DB ID - the first 4 numbers is sufficient (displayed after successful uploading of logs)
Regards,
Martin
Re: Duplicate DM Actions
Hi Martin, this is done, Database ID: database_backup_3.4.1.664_2023-05-30.56180432_martin@rootjazz wrote: ↑Tue May 30, 2023 10:44 pmif could send the database from this instance, it will probably be easier to see / confirm what happeningAlexiom wrote: ↑Tue May 30, 2023 12:12 pm One more update for now. To check if I had some kind of corrupted installation or data I tested this on another instance of TwitterDub I have that runs only about 70 accounts in a different niche. Exactly the same issue when I ran DM's across all accounts in parallel, existing global logs were shrunk/deleted during the process.
Can you submit your database please
TOOLS > CREATE DB BACKUP AND SEND TO SUPPORT
then send your DB ID - the first 4 numbers is sufficient (displayed after successful uploading of logs)
Regards,
Martin