Don't Reinvent The Wheel, Unless You Plan on Learning More About Wheels

The introduction to Head First Design Patterns exhorts us not to reinvent the wheel:


This is a companion discussion topic for the original blog entry at: http://www.codinghorror.com/blog/2009/02/dont-reinvent-the-wheel-unless-you-plan-on-learning-more-about-wheels.html

There are two sides to this coin, and in my recent experience.

On the one hand, there are folks who continue to patch 30 year olde codebases, just because they think it’s too much work (it isn’t, if you know what you’re doing) to do it right this time. So, they keep adding bells and whistles to square wheeled trikes.

On the other hand, the xml/XQuery folk are desperately attempting to make an old fashioned hierarchical datastore as agile as the relational database. They’ve re-invented the square wheel that was replaced 35 years ago.

The moral: only re-invent if you’re really, really smarter than the guys who did it first.

Moore’s Law. Double your workload every 11 months.

Moore’s case here is different. It’s not reinventing the wheel. Their product is/was Forth, so it was a technical / strategic decision not to rely on 3rd party code. Their core competency is the language and the compiler and how well the code runs, so it’s in their best interests to rewrite that particular wheel in a platform-specific, controlled way.

As others pointed out, Moore’s decision to rewrite Forth for every platform might not have been the best use of his time. While he personally may have benefited, it would have driven PHB’s insane.

If I’m just making a vanilla website that runs on any old server, I don’t start by writing a web server, I use a platform. If I want graphical buttons I don’t start with rasterizing routines, I start with the GDI routines or whatever is provided. I only step outside and do it myself when I’ve determined what’s out there is junk.

Maybe I need a web server on small devices. I’m writing the web server because I can’t or won’t use IIS.

Reinventing wheels does come at a cost. Library code has been debugged, and usually if there’s an issue, it’s because you’re doing something wrong and not the library. If you create the code yourself, you give yourself more points of failure, more code to maintain. You don’t want to go to the extreme of doing everything yourself because you otherwise spend all your time redoing what’s already done instead of making progress.

FWIW, Forth is embedded in a lot of places where it isn’t labeled Forth on the box. Also, Forth influenced a whole lot of designs, in ways you might never think.

I’m often reminded of the early days of the Macintosh, when someone said it would eventually take over the world. In truth, it did. It just wasn’t labeled Macintosh on the box, but Windows.

And yeah, I programmed in Forth, and have written various Forths and Forth-like systems. Some were toys. Some were reinventing the wheel. But some are embedded in places you’d never think of.

@ pumpitup Library code has been debugged, …

Sometimes yes, sometimes no. I am often amazed and astonished at the craptaculosity of some libraries. Yes, some open source falls into that category. The number of open source contributors is no guarantee of quality.

Matt pretty much hit it on the head for me:

Reinventing the wheel is fun for the developer and good for him (he gets to learn how to make the wheel, etc.)

But for the customer, 99% of the time, the original wheel is good enough and all you are doing is wasting their money/time.

Sure - re-invent away if it is on your dime. If you’re working on someone elses you’ve got to have a pretty lucid reason for why you’re re-inventing the wheel and get them to buy into it.

On a side-note, once you’ve seen one wheel surely re-inventing it is not that hard.

Reminds me of an old Joel on Software post:

And that’s where I learned a key lesson in software architecture: for your most important, mission critical stuff, you have to use a tool that is one level lower in abstraction than ideal. For example, if you’re writing a cool 3D shoot-em-up game (like Quake, around the same time period) and your key number 1 differentiator is to have the coolest 3D graphics, you do not use whatever 3D library you can find. You write your own, because it’s fundamental to what you do. The people who use 3D libraries like DirectX are using them because they are trying to differentiate their games on something other than 3D performance. (Maybe the story line.)

That was a different line of argument, though – he was suggesting that for stuff that really has to be completely perfect, you write it yourself, because then you can fine-tune it until it’s good enough. You argue that reinventing the wheel is good from a pedagogical point of view. I guess both are useful.

Sure - also related to

http://blog.codinghorror.com/when-understanding-means-rewriting/

and

http://blog.codinghorror.com/programming-is-hard-lets-go-shopping/

Is the reinvention thread-safe?
Who cares? Threads aren’t need in most programs.

Does it scale?
Who cares? Most code doesn’t need to scale, and when it does, you can always change it then.

Is it reusable?
Who cares? I’m making it for me, not anyone else. If I need to reuse it later, I can always change it then.

On almost every project I’ve ever worked on, the requirements were a lot easier than they first looked.

Interesting that you bring up forth. As an educational exercise, I did the same thing. I took http://www.annexia.org/forth (Jones Forth) and coded it up on an AVR microcontroller. What did I cover - Subversion, AVR assembler, threaded code and shift division. I’ve done shift multiplication but shift division was all new. http://www.tofla.iconbar.com/tofla/arm/arm02/index.htm

Think of replicating experiments done by others.

Here’s what happens when you don’t reinvent the wheel:

Can’t agree more - I recently sat down and created a REST library from scratch to manipulate data on Amazon S3 on Win32 without having to rely on the .Net library. I know that many coders out there would say Duh - the .Net library makes it a piece of cake to write SOAP enabled web clients, but I think I have learned more about HTTP protocols and socket threading from my (sort of pointless) exercise than I ever could by just calling .Net library code.

I cannot put a value on the knowledge I have gained from this one project, and I know that it has made me a better programmer in many ways.

Sometimes, reinventing can get you inspired to think about solutions in a whole new way.

I’m currently working on an existing custom app for a client. They’ve reinvented the wheel other the years. A lot. They’ve reinvented syslog. Very badly. They’ve reinvented SysV init. Poorly. They’ve reinvented message queues. Weirdly.
And it’s full of bugs. Unmaintainable.
I guess the lesson should be, you may reinvent the wheel IF you’ve looked at existing wheels and they didn’t work for you. Not because you weren’t bothered to do 5 min of Googling to see if wheels had already been invented.

I think you are ignoring one core aspect of the programmers mind: wanting to do thing Right. Clean. Proper. I often enjoy reinventing wheels that are already existing because I enjoy knowing that my wheel is clean, and round, and doesn’t have an adjustable size and 17 sets of tires and a complex AI that decides when to roll. I don’t enjoy using third party libraries that may or may not do what I want and may or may not do it in the way I want it to. I’m certain I am not alone with that sentiment.

Reinventing wheels is not always bad, especially not if you are programing for fun instead of productivity. As long as reinventing the wheel is a conscience decision, and not lack of knowledge about existing wheels, it’s perfectly fine to do so.

Having said that, I think I could do without programming another linked list in my lifetime :wink:

I always want to reinvent the wheel, but you gotta know when it’s appropriate and when it’s not.

I guess that you should reinvent the wheel when the wheel wasn’t invented properly yet. We’re all coders, we’re good and we should be able to know when the wheel is broken and when it’s done to perfection. If it’s perfect, you can try reinventing it, but consider using the perfect version.

Hmmm, interesting, interesting, but it all depends on the context. If you are trying to learn how something works under the covers, reinventing the wheel can be fun and rewarding. However, if you write code for a living (or is that professionally?) then this is likely to be a bad thing because:

a) Depending on the size of the wheel, your (teams) implementation is likely not to be as good as another team that has been doing the same thing for years, refining the wheel so it runs smoother and smoother, etc. While this is your first time to the rodeo, prepare to fall down a lot with your square wheel.

b) Writing anything of any size is hugely risky and massively labor intensive in our software development world. Writing “quality” code is especially hard, which is why you see so little of it. It is easy to make a prototype – really, really hard to make it industrial strength – cost lots of time and money.

c) Customers are not interested in you reinventing the wheel, they want it done as soon as possible, as cheaply as possible and as best quality as can be had, as long as it does not take too long or cost too much. See b).

Have fun!

Wow, this post came at a highly apropos time for me. A friend and myself are just beginning a video game project, searching out engines to make our lives a little easier. I’m utterly new to programming, but have a lot of experience in other fields of video game design.

Every single time I sit down to write a script, or start to program, I end up re-inventing the wheel. Was just talking with him about this last night while dredging through libraries and comments, looking for the elusive wheel to fit our car.

I want to reinvent it, while at the same time I have no idea how to start. Our game does the wheel differently, from what I have seen (lots and lots of video games under the belt :D) and I WANT to do it better.

So as I said, very apropos.

I have recently been reinventing a .NET service bus (see nServiceBus, Mass Transit, and Ayende’s reinvention Rhino Service Bus). When I first grabbed nServiceBus, I had a hard time figuring out why all its pieces are built the way they are. Taking Udi Dahan’s (awesome) SOA course certainly helped there, but I figured that building a service bus myself was the best way to really understand it. With that knowledge, I might just abandon what I’m doing and go back to nServiceBus or one of the other existing solutions.

A secondary motivation is, as J. Stoever said above, that I might want to remove a bunch of the flexibility, in order to enforce my own opinions of how it should work on the users of my bus.

I guess I’m reinventing not just the wheel, but the whole bus.

Well well, what a load of fatuous cockwaffle. Particularly this :
Don’t reinvent the wheel unless you’re one of the 0.05% of people smart enough to get it right

Hey, sweetcakes, for your information I personally AM one of that percentile that you’ve pulled out your ass. I am the annoying bastard who can sit down and casually, over a couple of slow afternoons with lots of smoke breaks, sit down and code complicated, correct, software modules in some language that I only heard of yesterday.

I didn’t get to be the coding ninja that I am by using some whiney freetard’s homework project of a library every time I’ve bumped up against a hard problem that isn’t solved in the language or framework I’m working in. You bone up on the problem domain until you live and breath it, and then you write your code. If you couldn’t code it, you certainly don’t understand it sufficiently to be throwing implementations of it around. If you aren’t capable of doing this in a manner that provides value to your clients, that’s a failing on your part, and your clients should come talk to me.

Let me give you some clues you can use :

  1. if you think that only 0.05% of coders are competent to do this stuff, you’ve got a serious problem with your own argument. If I wet my pants every time I see some hard maths and go pick up a library, I’ve got little chance of finding a good one, since the vast majority of contributing developers will be outwith that percentile, and the majority of code will be crap.

  2. Whatever the percentile, what makes you think that we’re going to be packaging up and distributing our shit and sharing it with you ? What would be the motivation ? Fuck the open source love fest, because we don’t play well with others, and there certainly isn’t a financial incentive, being better than you is our USP.

  3. If you honestly believe that all the problems a coder will come across have been addressed, solved, understood, and neatly packaged up for everyone to share, you probably spend to much time drawing UML models in crayon and drooling into an HTML reference. Get a real job.

The problem in software today isn’t that there are too many wheels, it’s that there are too many shitty wheels. Certainly it’s possible to reduce the rate at which shitty wheels are created by reducing the absolute rate of wheel creation, but this just skirts the problem.

The solution should be, rather, to produce higher quality software and higher quality parts. Moreover, if there were higher quality components out there (and there are starting to be), people will use them naturally. An excessive abundance of quality libraries and wheels out there is a problem that every software engineer would be eager to have.