Hi, Martin
after adding this function, it will take 5-7 hours to make 30-35 likes
https://imgur.com/SxPWgWS
https://imgur.com/ncBMOiV
Cleaning out and minimizing logs. Application currently crushing 8 core VM.
- martin@rootjazz
- Site Admin
- Posts: 34627
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Cleaning out and minimizing logs. Application currently crushing 8 core VM.
Please in future start a new thread for your issue, as it's not related to this thread.Emin wrote: ↑Sun Jun 04, 2023 2:23 pm Hi, Martin
after adding this function, it will take 5-7 hours to make 30-35 likes
https://imgur.com/SxPWgWS
https://imgur.com/ncBMOiV
As to the issue, no idea with logs
Can you send raw logs please:
right click the action > RAW LOGS > copy and send me the contents of the file that opens.
Code: Select all
support[at]rootjazz[dot]com
HELP > LOGS > SUBMIT
then send your logs ID - the numbers are sufficient (displayed after successful uploading of logs)
For future reference, please, read through this post and submit your issue accordingly. Not everything will be relevant to your issue, but have a read as it gives an idea of the information you can provide to help me help you as quickly as possible without delays in having to request additional information
https://rootjazz.com/forum/viewtopic.php?f=23&t=1634
Regards,
Martin
Re: Cleaning out and minimizing logs. Application currently crushing 8 core VM.
my problem is related to this post
I didn't have this problem before adding this feature.
id_logs 38562
I didn't have this problem before adding this feature.
id_logs 38562
Re: Cleaning out and minimizing logs. Application currently crushing 8 core VM.
I think it's related to this thread:
https://rootjazz.com/forum/viewtopic.ph ... 0&start=20
Martin was able to resolve an issue of global logs being destroyed while multiple threads were processing. Now the global logs stay intact and duplicate actions are correctly identified.
However, I'm seeing the same issue as you, multi-treaded performance has tanked. If I run 10 threads, each thread takes nearly 10 times as long to process.
In the meantime, I guess you can roll back to an earlier version...
Martin, can you assist in troubleshooting this?
https://rootjazz.com/forum/viewtopic.ph ... 0&start=20
Martin was able to resolve an issue of global logs being destroyed while multiple threads were processing. Now the global logs stay intact and duplicate actions are correctly identified.
However, I'm seeing the same issue as you, multi-treaded performance has tanked. If I run 10 threads, each thread takes nearly 10 times as long to process.
In the meantime, I guess you can roll back to an earlier version...
Martin, can you assist in troubleshooting this?
Re: Cleaning out and minimizing logs. Application currently crushing 8 core VM.
yes, if I run only 1 thread, then everything is fine, but the more I run tasks, the longer the process takes
Re: Cleaning out and minimizing logs. Application currently crushing 8 core VM.
Did some testing. With my configuration, running the DM process, the average time for a job to run doesn't take too much of a hit from 1-10 threads. But at 20 threads average job takes twice as long to run, so throughput is the same as just 10 threads. But I'd love to run 200 threads
- martin@rootjazz
- Site Admin
- Posts: 34627
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Cleaning out and minimizing logs. Application currently crushing 8 core VM.
ok so you updated and the issue started.
Your logs are strange, something is looping loading the global log... But that wasn't changed in this update, just hardened the concurrency code in accessing the global file.
Cannot think why it's taking 10x as long...
- martin@rootjazz
- Site Admin
- Posts: 34627
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Cleaning out and minimizing logs. Application currently crushing 8 core VM.
I'm guessing it was you I told in the email, I'll need to have a think and completely change how global logs are accessed, as the current method is not fit for purpose at scale.Alexiom wrote: ↑Tue Jun 06, 2023 5:51 pm Did some testing. With my configuration, running the DM process, the average time for a job to run doesn't take too much of a hit from 1-10 threads. But at 20 threads average job takes twice as long to run, so throughput is the same as just 10 threads. But I'd love to run 200 threads
I have an idea how it can be done, but will need to code it from scratch. Please bear with me (and give me frequent nudges to hurry up and get it done)
Re: Cleaning out and minimizing logs. Application currently crushing 8 core VM.
I only do like,dm and dm watch followers,quite a long time, more than a year, and this has never happened ... already 300 accounts banned
- martin@rootjazz
- Site Admin
- Posts: 34627
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Cleaning out and minimizing logs. Application currently crushing 8 core VM.
ok so I had some time to get it done. Should be much better now (if the issue is what I thought it was).
I'm thinking the same technique could be used to on all processed_logs, not just global logs, but would increase memory usage considerably
I'm thinking the same technique could be used to on all processed_logs, not just global logs, but would increase memory usage considerably