My Scaling Hero

If you haven’t known, check this out.

[duplicate of 2nd link in article]

he’s the no.1 adsense user in Canada, making 10million a year, which is way more enough to pay off his bandwidth bill. :slight_smile:

I disagree with the last quote. There were definitely hacker communities in the early days of programming (see Richard Stallman’s book Free Software, Free Culture for examples) that encouraged sharing of code to a great degree. Heck, Stallman even started GNU following frustration from being unable to debug a proprietary printer driver, which was pretty much the only proprietary thing in the entire AI Media Lab, operating systems included.

Another thing to point out is that he is serving all those images through a CDN (Akamai). While very expensive, they do remove the load, space, and bandwidth from your servers

From highscalability blog: IIS arbitrarily limits the total connections to 64,000 so a load balancer was added to handle the large number of simultaneous connections

WHAT?!?! And people actually pay for that?!?!

No caching functionality in ASP.NET is used. It is not used because as soon as the data is put in the cache it’s already expired.

I’m thinking this is an example of doing it wrong.

There are so many ways/technologies/techniques for building websites it’s staggering … least to me it is.

It’s funny because the very first asp.net app I ever wrote simply had a server tag for returning the page generated server side. I thought it was totally hack at the time, but maybe not so much?

Then later on when I had to build some sites for a customer (and this is without, really any other experience) I used the ASP.NET server controls and whatnot for user entry and all. I really didn’t have any time (as I was the sole dev) to learn some of the MVC/jquery-AJAXy style stuff … and while the site works and is pretty solid, I read articles like this (and some of these comments) and just go Damn, 101 ways to skin a cat indeed…

Scalability worries me quite a bit. Very soon, me and two co-workers will be developing a web app (using C#/ASP.NET) and have very little experience in doing so as efficiently as possible.

I’m not sure our idea will directly get anyone laid, so it might be doomed to failure.

Did you mention he makes around half a million dollars a month in adsense from the site? The limited hardware could also be done in the linux world by simply avoiding apache and using other far better (free/commercial) replacements. But it’s the adsense that blows me away at $10 million in income per year.

This is very comforting as I am planning to scale my website which also runs on asp.net mvc. It is a hosted microblog platform and it is still in private beta and less than 2 weeks old and I already have 200+ sites created. My problems will be less than stack overflow because this is a paid service, so the growth will be limited. If it was free like twitter then I would run into those kinds of problems very quickly. The site I am referring to is www.yonkly.com

BTW, I am using Amazon EC2 hosting, so I can quickly provision new servers and do DNS round robin if there is a huge surge in traffic. Scaling the database would be another problem. But being on EC2 is very comforting.

I’m still doing everything myself except for customer service.

Current numbers are.

60-65 Million Pageviews per day, 4.4 million per hour peak.
72 million visitors a month
3 load balanced Windows Server 2003 x64 web servers with 2 Quad Core CPU sits at about 20% cpu at peak, 2 Quads… Makes it easy for me to take servers up and down without worrying about crashing.

Sites running on 5 DB’s now, between 64 and 128 GB of ram, Got a huge 30% increase in traffic coming on Dec 26th same as every year. Every page on the site has a bar on images on it… To create that bar a radius search is done along that uses your history as well as preferences. Its not possible to cache anything as its true personalization and each query depends on the previous one. Same with all the searches.

Over 130,000 Concurrent users at peak.

500M images per day.

Asp.net for the most part is irrelevant when it comes to scaling, because it uses so little resources. I use StringBuilder, Response.Write and access the DB. C# is the last thing I think about when it comes to scaling issues, its 99% the DB.

Jeff -

In the 1970’s many of my colleagues would not run a program unless they had read the source code and compiled it themselves. Those who wanted to share their work put the source on open display - it certainly was not like your love letter metaphor.

-Lepto

@Starburn: A database won’t scale beyond a certain point, but where is the difference of throwing better hardware at it compared to running more web servers? In both cases you must buy hardware. The only difference is that in one case you replace hardware, in the other case you add hardware. However, no matter how many webservers you run, if they all share the same data, you’ll need a central database again and this will sooner or later always become your weak link.

@BuggyFunBunny: Youngster :slight_smile: Well, I’m slowly starting to group out of this term. At least my cards don’t say Junior Software Developer anymore, just Software Developer. Lets see how many years to go before I’m a Senior. And all I know about LAMP I learned from the best (unfortunately one of them already left the company). When I started, my first project was a grown, old, web-based document management system. The task was: Make it run faster. This was the first time I was doing something meaningful with PHP and the first time I really had to understand SQL Selects in all their beauty. By limiting the amount of SQL statements (2 statements per page viewed instead of 31 statements per page, still with the same data in return) I was able to speed up the system dramatically. From that day on I really studied SQL statements to find out how much more work you can actually already do in the database, since it is also much nicer for a programmer to just create a statement of how the result should look like instead of getting a huge blob of information and having to manually write code to get it into the desired state.

i’m not sure if it’s been said, and it definitely doesn’t belong on this blog (except that maybe you care, since you said a few things about security).
plentyoffish has the WORST security in the history of the internet. not only does it (have to) have a completely unencrypted database of the passwords of every user, it sends em out WITH EVERY SINGLE EMAIL IN PLAIN TEXT.
why would anyone want their password sent to them every single month ? i do not know.
if one was in the habit of using the same password for everything he signed up for on the internet (not a stretch of imagination here) he could not read his email with someone standing behind him, ever.

ok the rant is over.
The problem with free is that every time you double the size of your database the cost of maintaining the site grows 6 fold.

that is a point to remember. and i think that’s a point advertising companies should take into consideration. the problem with this is … you hardly ever get negative growth. so eventually you will have to charge people for using your site since the ad revenues don’t cover the cost anymore. which would be bad for everyone involved, the advertising company (or the sites you’re advertising for) yourself, and the users.

I think the simplicity desgin of Plentyoffish, gives users a feeling of trust, and belonging. After all, most of us (singles) are living in a messy room. When it comes to websites deals with personal matters and life, I much prefer the low key, causal design of Plentyoffish, just a sense of belonging.

Ross Lin.

Jeff,

I’d like to start a free budget/personal finance site and will need plenty of horsepower and storage. I’m definitely a nubee, so how does one get started in terms of hardware infrastructure? Do I need to get a dedicated IP from my ISP and obtain my own servers? Do I go with amazon or Sql Server Data Services/Azure or something like that? Is the latter services free or a discount offering available to non-profit/free services such as the budget/personal finance site I’m looking to start?

Every time I feel like I’m going numb and going into demotivational stage, I look up at Markus’s work and refill my lunges with new energy.

Awesomeness!

Its too bad Marcus wont share more information to help others with their web development and programming quest. I know many have helped him along the way, I am not sure why he would not want to return the favor. I would think he could at least tell someone new where to start their learning, how to optimize code, what technologies he thinks works best and what areas they should do research for the future how to figure it out for themselves how to get a high traffic automated web system up for their online ideas. Example

1.) Asp.net - Learn these functions of it, don’t use this part of it and optimize your code by doing this (Study this other technology that I am thinking about changing it to for this reason possibly? Future plans?)Tips on automation and optimization. Should we think about C# or other future Microsoft technologies?
2.) Web Servers - Recommended Hardware, Windows 2003 64bit, this hardware, this setup and configuration and use it this way
3.) SQL 2005 (Or whatever) - Recommended Hardware, Windows 2003 64bit, What version, how setup, tips on utilization, use this way for reads and use this way for writes, how to optimize for high traffic, Etc.
4.) Bandwidth and other network optimizations?
5.) Good resources you used online to help you figure stuff out??
6.) Any other helpful tips on AI and other stuff you use?

You are rich now! This is one way you can share your wealth in knowledge without having to spend a cent. Just give us enough to figure out the detailed creation ourselves?

Or create a framework we can buy or something. Just help a brotha out!

Bless us oh Marcus our Friend! :slight_smile:

I think the simplicity desgin of Plentyoffish, gives users a feeling of trust, and belonging. After all, most of us (singles) are living in a messy room. When it comes to websites deals with personal matters and life, I much prefer the low key, causal design of Plentyoffish, just a sense of belonging.
http://touravus.ru

I think Mr. Frind will not share or discuss his secret…The reason it’s business.

at first i thought it’s about the image rescaling issue, which is something PoF.com clearly needs to work on as well…

Well if Microsoft reads this they should donate him some resources in exchange for the advertising he’s already doing them. Putting these numbers on a fact sheet for ASP.NET wins the competition hands down.