Planning for the future - Optimization with Large Databases and Virtual Machines

[snowleopard]

Hi Martin + Community,

I am wondering if I can get your advice on how to plan for the future.

Currently, I run SCM on a Virtual Machine, an EC2 via Amazon. It has an Intel Xeon Processor 2.5 Ghz, 4 Gigs of Ram and 30 gig hard drive.

I was happy with the performance, until I started to build up Proxy Accounts with a second Database instance. Even with only 70 Proxy accounts on the second database, performance slowed down to mind numbing speeds. I click a button and wait 8-10 seconds for the click to register.

It is my goal to have a healthy roster of approx 1k proxy accounts, along with around 25 normal accounts that I manage. What I am trying to decide is how can I move forward and ensure fast processing and room for growth?

Would the best way be to:

A) Upgrade to a new Virtual Machine that is much faster and with a bigger hard drive and more ram?
B) Divide the accounts over MANY virtual machines and ensure no more than a single database runs at a time.
C) Divide the accounts over FEW virtual machines and *TRY* to run 3 databases per machine (the maximum).

Obviously with more than three installations of SCM, I would need to purchase more licenses. That's totally fine. I'm just concerned about PERFORMANCE. How can I ensure that I can continue to grow and maintain fast speed?

Any help / advice would be GREATLY appreciated!!!

Thank you.

[martin@rootjazz]

you shouldn't be getting a slow down like that.

what version are you on?


What actions do you do? when exactly does the slow down occur.

How many accounts per instance do you have?


You can run as many instances per machine as you like.

So installing on 3 machines, then running 10 instances on each instance, would mean you can run 30 instances on your single purchase

[snowleopard]

Ok well good to know that I "shouldn't" be experiencing a slow down like that. I thought it was weird my self.

Version of SCM was Beta 3.039

In terms of the actions, all pretty standard stuff:

Messaging
Commenting
Following / Unfollowing
Favoriting / Unfavoriting
Group Shares
Reposts
etc.

The slowdown would always happen when opening the Second Database. The first database was less than 10 accounts. The second database was more around 70.

As soon as the second instantiation of SCM was up and running, slowdown on the EC2 was immediate.

I couldn't even imagine trying to run 10 instances of SCM with those current conditions...! But that would be ideal!

Maybe I just need to beef up the EC2.........

Hmmmmmm

[martin@rootjazz]

Opening multiple instances of SCM has no issue on my server VPS. It isn't that big a VPS either. I have most our software open in 2-3 instances, so about 8 - 10.

no slow down.



I use / recommend Hostwinds
https://rootjazz.com/vps



You can use the coupon
25% off (not sure if is recurring or just a one of - probably a one off
rj25offvps


Black friday is coming up and they usually do 50% recurring discounts, so might be worth thinkign about changing provider

[snowleopard]

Oh wow, that looks great!

Looks like I could build a much faster setup and for less money.

And I love that the pricing is very upfront. Trying to decipher Amazon EC2 pricing is tough.

Thanks for the tip!!

[martin@rootjazz]

yeah AWS in general is confusing. Never heard of anyone using it as a windows VPS for bots before. Great for at scale websites as easy to add to, but that isn't really something you need to pay a premium for with regards to running bots

[arcan210]

I tried AWS for my VPS and had a very similar experience. Hair-pulling frustration because of slowness. If you havent already bought a new VPS, I recommend WebKeepers.com.

You get literally NO support from them. They assume you know how to manage a server. If you have major issues, the only thing they will really be able to do for you is restart a frozen system or reset to the factory state. However.. with that said... they are the cheapest and their boxes are blazing fast. Ive got 3 different instances of SCM running 24x7, one with hundreds of proxied accounts that I use for retail jobs from DominatingTheCloud.com. Those accounts are all churning through giant lists of random activity to make them look as realistic as possible. Other instances are for audience growth for a dozen or so accounts that are also doing a lot of daily automated activity like follow/unfollow... group shares... likes, reposts...etc...

Basically, theres no reason that a box with the specs you provided shouldnt be able to handle SCM in the way you describe. My VPS from webkeepers isnt as highly specced as yours and I dont see any of those probs. Time to ditch amazon, Id say.

[snowleopard]

Right on, thanks for the info Arcan210. Good to hear I'm not alone with the frustration!! Going to switch to something new imminently.

Will check out WebKeepers and compare them to HostWinds. HW is doing a killer Black Friday deal right now so.....

Cheers!

[martin@rootjazz]

That reminds me I need to get a new VPS server, will get one this weekend.

50% off - although this year is only 18 months rather than for life. But can just get another one next black friday