I would still say that the root problem was that you were using the wrong solution (i.e. storing and importing data from client), but that’s difficult to prove in theory.
ask yourself: am I working at the right level of abstraction?bold and italic!).
RUNTIME is essentially the same as PORTABLE. CONTAINER, as the name suggests contains various portable entities, including PORTABLE.
In the process to build RUNTIME out of PORTABLE, my system typically just pass the data it doesn’t understand so the next stage (let’s call it “link” stage) has the data in place.
Or maybe not?
I spent a whole day trying to figure out how to properly “link”, with code getting grosser and grosser. At the end of the day, it was so magic I had to review the “linking” rules; turns out that the line count goes 400% or so. For something designed for non-programmers this meant sure fail.
The day after, drawing on paper the pipeline giving me PORTABLE, I figured out an important aspect I overlooked:
Just because RUNTIME contains the same data than PORTABLE (+ the runtime optimizations) it doesn’t really have to contain the same info! It turns out that the rules are not needed at all if the high-level “link” info is moved back to “low level” PORTABLE representation.
Abstraction may effectively reduce the need for informations. In this case, it increased the need.
Reviewing the whole pipe, I had tracked at least 3 important quirks, 2 of which were design byproducts; the last was an implementative issue.
So, I reworked PORTABLE to include the “link” information (currently, 2 special bytes). Problem solved. No special rules required. No complicated “link” required.
Long story short: abstraction is powerful, and it is both ways. Some apparently difficult problems have surprisingly trivial solutions at a different abstraction level!
No, I wasn’t working at the correct one.
Once again, I found a solution coming from LOW abstraction as opposed to higher.
I had the unfortunate idea to add a tag to it and it went severed.
When I read “NO HTML” i think than HTML isn’t supported because it is escaped. I don’t really figured out up to now that it actually causes messages to be severed.
So, no HTML… ok, but at least some input sanification/escaping?
What’s wrong with bold and italic really?
…
this is about that blog cliches thing.
i must say i agree with you on some 12 points of yours, but the diary thing was a bit troublesome. of course, the reason why some web space is given to you is undefined, and i am looking for answers still. but you have every right to utilize that webspace of yours as your diary, and when i say diary, i do not mean what you did as a routine, but anything that contains your thoughts.
hope a quick reply
vivek.
ps:i am aware of all the disclaimers you put in that post, but i think i may get a desirable answer from your side. you seem to be an ardent blogger, by the way.
One thing that isn’t mentioned enough is the vital question:
“HOW LONG DOES IT NEED TO REMAIN SECURE?”
To me this is more important than whether something should be encrypted or not. A password, for example, may not be stored at all but must still be transmitted in some way, and I for one don’t want that done in plain-text.
Counter to popular belief XOR ing or even ROT13 are forms of encryption. VERY basic, and very easy to crack, so now they are only used to post answers on blogs and web pages so that the answer isn’t immediately apparent and the reader can have a go at solving the question.
So - in that instance the data doesn’t need to be encrypted for very long at all, which is why XOR and ROT13 are appropriate. The longer the data needs to remain private, the longer it should take to unencrypted the data. The time taken to unencrypted given the CORRECT key is directly related to the time taken to crack the encryption with ANY key.
So what about passwords? Well - if updated regularly an encrypted password only has to remain secret for a couple of months - maybe a year or two to be safe. I’m not suggesting they should ever be encrypted - they should be hashed, but even then there are reverse-hash mechanisms, but the variety of outcomes would still take a while to test. So in that case higher level of encryption should be applied - say an encryption of the password, then a hash, then an other encryption.
What about private data like banking details? I believe that this sort of information should remain private for at least the lifetime of the person. Can we really guarantee this type of security using current technology, when computers are roughly doubling in speed every 18 months? What happens when quantum computing comes on line?
Even the current standards don’t offer much protection in the long term.
DES is no longer even considered for encryption, but each password had up to 72 thousand billion possible combinations. But DES is now calculated quickly, so it takes very little time to try all of these combinations. DES is about 35 years old, and computers now make short work out of it. Any old data that was encrypted using DES is now insecure.
Triple DES is closer to 10 years and it wasn’t long ago that Triple DES was considered rock solid, but now it is considered less than optimal. Yet there are 370 Trillion Trillion Trillion Trillion combinations. And that is for every block - so with appropriate salting it was considered almost impossible crack Triple DES. But with today’s computing power, and the ability to grid zombies these encryptions can be cracked.
Triple DES is still too hard to crack quickly, so it is still used in the payment clearing technology, but it can be cracked over time, and so shouldn’t be used for stored data.
I think this is one of those areas that you need to have the knowledge to ask the right questions at the outset AND damb well make sure you have the skills to answer them correctly.
To paraphrase Simon above, the biggest lesson is that nothing will help you if you genuinely don’t know what you are doing.
Your clippy also needs to tell you that on 10.10.2004 you shipped code that uses AES 128bit which was appropriate at the time but is now insecure. There are 214 copies still in the wild using insecure encryption. Obsolete only works when you recompile / rerun static analysis like fxcop.
All of the below is given that a certain developer is even marginally aware of good techniques. That’s already cutting out 60% of people who write code.
From my experience, whomever in MS created the API’s didn’t follow the BCL team recommendations. Have a simple API for basic tasks, and provide a in-depth API in addition to a straightforward transition to using the in-depth API. But as mentioned, very little in crypto is basic, and MS’s own implementation and defaults don’t help at least for me as I only ever write web farm server software.
MS appeared to create crypto that was heavily tied to the logged-in user, which works great for Windows Forms type software, but not so great otherwise (and I still don’t trust the implementation of encrypted files in windows if it ever loses my private key). I would never create a backup of an encrypted file.
The documentation, when you could find it, was horrendous. I remember trying to do some of it back in COM+ days, and lets just say i must be an idiot because i never could figure it out, or at least wasn’t confident it was done correctly. Coding by coincedence indeedillydoo.
(In the end I downloaded reference code for a simple symmetric encryption scheme which was good enough when all other factors were weighed in. I could test it. It was straightforward. Job done in less time than I spent reading CryptoAPI docs.)
Playing the devils advocate …
- Crypto in .net is an API.
- I might not like or be able to understand the decsisions made by BouncyCastle, etc (or have the time to)
- My team/company may forbid external / 3rd party libraries (it is a dependancy)
Too many developers scratch their heads for how to actually get crypto to work and deployed, and when presented with a deadline we have:
- proper crypto-api. (too) hard to use. don’t really understand. my code will probably be buggy anyways
- use some other library. too long to evaluate. are they secure? can i see the code? What do I do when they update their code. Do I always need to update?
- do something basic and write your own code (too many XOR or hex-encodings)
- no crypto
You advocate ditching 1 for 2, but in my experience people typically go for 3 and 4.
Crypto is a moving target, so any decent API needs to take that into account. What happens to old data? Old keys? Revoked keys? Do I care? I don’t care. Lets go shopping.
So what do I think? Forget API’s.
Encrypting the communication channel is better than manually encrypting the data in the database. If someone has your database or local file access, you’re already stuffed.
Protect your browser. If another website can read cookies or know anything about other websites you visit, you’re toast.
Protect your OS. If someone can install a keyboard sniffer, you’re toast.
Protect your users. If someone can socially engineer details out by calling support staff, you’re toast.
Spot on but off the mark.
thx
Phillip, you posit what sounds like an interesting idea at first but upon closer inspection, isn’t very interesting at all. Most people are not working on a problem thinking “I need this to be secure for exactly 2 weeks but after that I don’t care.” Your example of ROT13 is not really secure at any point so I’m not sure what you mean by that. It’s more interesting to think about what the sensitive data means to you. If the data is only worth about, say, $1 million (perhaps because of a lawsuit resulting from a breach) then spending $100 million on securing that data may not make sense, depending on other factors as well.
The idea that encryption algorithms become weakened over time is fine but this is only interesting in your example if people built applications with the desire to never change them over 10 years, which is not an interesting case either as most software that goes unchanged for 10 or more years becomes vulnerable for other reasons besides the viability of the encryption algorithm.
@Rodger, your view of not caring about encrypting data because if they can access it, you’re already compromised falls down. If some data is accidentally exposed because it was not encrypted, do you really want to pay the data breach penalty just because you have this view? For a trivial example, if you store passwords in a User table, do you really want to risk exposing plaintext passwords because of a bad SELECT statement or some other mistake just because you subscribe this naive view? I don’t think its worth it, personally. It’s probably easier to just do it properly in first place.
Charles:
I take your point, but I disagree.
I agree that programmers don’t think that “this needs to be secure for x amount of time”, but they SHOULD. It directly impacts the performance of their program AND the long-term security of their data. The very best security mechanisms are SLOW, and the very worst are fast - it IS a scale. It’s not about dollars, it’s about time. It takes time to encrypt the data and it takes time to decrypt. The longer that time, the better the encryption but the worse your program performs. That’s why the question is so important, and that’s why it is important to get it right - especially if you have a lot of data.
Also - this is NOT about updating your processes over time, this is about data you have RIGHT NOW that gets stolen/acquired RIGHT NOW.
I recently went to a fraud presentation by a top police official, so I found out some of the practices of the criminal element in our society.
One of the things the criminal element are doing is collecting information that is available to them now, even though they can’t crack it now. Storage space is cheap. They have dedicated computers attempting to crack the information, and as time progresses they are more and more successful at getting historic information.
So when I say “how long do you need to secure it” - I mean, assuming someone gets the encrypted data from your storage NOW (and if that wasn’t a possibility the data wouldn’t be encrypted) how long before they can get to that data easily.
Cases of sales staff stealing company databases isn’t unheard of - and much of that information, such as customer contacts, will still be of interest for at least a decade.
In this country (Australia), CDs and Laptops of top people from government departments, including military, have been stolen/lost. We better hope the developers of those applications DID NOT intend on updating the program and encryption every couple of years. We better hope they made the encryption very rigorous RIGHT NOW.
RE ROT13 - anything that obfuscates the original text is a cipher, even if it is so easy to get back it can be done in your head (think of those code rings).
API’s don’t protect you from bad programming. It’s a red herring and doesn’t detract from my position.
My overall point was the individual program is not the right level of caring about encryption.
“The very best security mechanisms are SLOW, and the very worst are fast - it IS a scale.”
One of the key factors in the AES selection process was performance in various software and hardware settings. So you want to tell us that they’ve finally selected the candidate with the worst security for AES?
The only encryption algorithm provably uncrackable (Vernam) is the fastest available – it takes one operation per byte for encryption/decryption, generation of the key material aside.
API’s don’t protect you from bad programming. It’s a red herring and doesn’t detract from my position.
My overall point was the individual program is not the right level of caring about encryption.
Rodger on June 14, 2009 7:36 PM
I’m never working at any level of abstraction other than abstraction I have written myself.
So you do all of your coding in assembly?
Actually assembly is an abstraction for machine code, and after that the question is can you trust your CPU?
There’s nothing worse than working with 3rd party software that doesn’t work.
Everything is all well and fine when things go right, but when that 3rd party library f**ks up you’ll wish you had written the code for yourself.
Maybe a more instructive blog would have been to write an analysis of how much time it takes to get a 3rd party library working, with all the necessary work arounds, lost development time etc against the time taken just to get straight to the spec, implementation and testing.
I have to agree with @Jackie.
Even if you do go with a 3rd party you aren’t guaranteeing that you are solving the problem. Once you have that dependency then you need to trust them to do the right thing.
At least with an OSS vendor you can tweak it yourself if needed but then you are back right at the start where you have to be able to understand the code yourself.
Gibson’s crusade against raw sockets isn’t just a wrong analogy, it also wasn’t holy - it was crying wolf, overdramatizing, and generally pretty pesky.
Raw sockets required admin privileges. When you have that already, nothing stops you from loading a driver and sending raw packets that way - more work, but no problem at all. If Gibson had had half a clue, he wouldn’t have shouted his mouth off to make MS remove raw sockets, but to make the default account in XP non-admin.
If I am paid to reinvent the wheel, then I will do so.
Raw sockets required admin privileges.
… which is the default under Windows.
When you have that already, nothing stops you from loading a driver and sending raw packets that way - more work, but no problem at all.
It’s quite a bit more work doing it that way, and given the state of malware when Gibson pointed out the problem (fairly primitive user-mode stuff), this was a very real concern.
If Gibson had had half a clue, he wouldn’t have shouted his mouth off to make MS remove raw sockets, but to make the default account in XP non-admin.
How about you do it for him? Good luck with getting that change made.
Okay - you are right, I should have been a little more explicit.
I took as given that itterations (key bit size) is the greatest security factor, after that (assuming the same bit size is acheived), then slower algorithms are more secure.
The security of encryptions comes by the number of possible iterations of “keys” multiplied by the time taken to try each iteration. For instance, an encryption that takes 10 mins to try when there are 1,000 possible keys is way more secure than an encryption taking 10 milliseconds to run with 100,000 possible keys.
So you are right in that it isn’t “JUST” the algorithm. But given the SAME key length (in bits), then a faster algorithm is less secure.
Let’s say that Triple DES runs slower than AES, then bit for bit it would be more secure than AES.
The benefit of AES is the key size of 192 or 256 bits to DES’s 56 bits or Triple DES’s 168 bits.
That is – there are at least 16,777,216 more combinations to try using AES compared to 3DES. Even if it is faster to encrypt and decrypt, it isn’t 16.8 million times faster. Use the full 256 bits and it leaves 3DES and DES in the dust as far as security goes.
Even so - as a general rule, the implementation of encryption that SLOWS DOWN the encryption/decryption process is more secure given the same number of itterations because… it takes longer to crack…
Triple AES, for instance, will take you three times longer to encrypt AND will increase your bit key length three fold. This will take you longer to process - especially large data files - but depending on your requirements may be what you require.
And that’s where it comes back to my original point, you have to first determine how long you need your data to be secure. The encryption process you select should meet that requirement.
Oh my god… What a load of selfinvolved nabcake BS.
I think you should clearly stop coding because you are giving us all a bad name.
Do it. Do it now!