This is a mini post to show how MaidSafe uses hashes to identify data after processing it in a way that’s invisible to end users. I hope it is at least a little bit useful. (note the use of encryption to obfuscate and later I may get into a debate about why we consider it only for obfuscation and creation of non repeating data). This is relevant to the previous post, where we describe hashes and their use for proof of retrievability and corruption free data.
How to make data safe?
Ok, to make this a shorter entry, I have embedded a video Shona in the office did for us to explain self encryption. It confuses many people, but it quite straightforward.
It should be noted, this process is a very small part of the “securing data” story. This will most likely secure data beyond any known decryption ability known today, but, and a big but!
This is logical security, what I mean is where do you store the actual data? On Dropbox, Google, some other provider, on your hard drive etc. Well no!
None of these give what you want, physical security! Physical security (to me) means that nobody can know where your chunks of data are, so this means that this data has to be scattered to the 4 corners of the planet, on many machines, owned by many people, non of whom know what they hold or can disable or corrupt it in a way that affects the network copies of the data. It also means nobody can ban access, monitor or affect your relationship with your data in any form. This data is yours, it belongs to nobody else and there is no logical reason that anyone gets between you and your data, none whatsoever. This fundamental issue lies at the heart of MaidSafe and it’s design. It is not something people ask for, most likely because they do not realise it is possible. It is possible, it works and its coming to everyone.
Why is it needed?
Well now we can start seeing the need for not only logically secure data, but also physical security. This about this for a minute, when in history of computing could we have private, physically secured data, accessible across the world? A mechanism where you can log in or access privately and without any third party get your private data securely. It is amazing, but never is the answer. You could argue, use truecrypt, backup tapes, copy to off-line and on-line backups etc.. All of which are hard, leak information about your actions at the very least and most importantly depend on great technical knowledge.
This is where you need an autonomous network geographically distributed and on peers that are monitored, not by humans, but by the network. This allows people to physically damage chunks, but the network spots that very quickly and creates a new copy and at the same time de-ranks your machine. Nothing to gain, except loss of safecoin earning now. So vandals can happily vandalise, they can use their vandalism energy on that task, that’s fine.
Without an autonomous network this system of protection is only 50% of the job, only logical encryption, worse than that the upload and downloads are monitored in on-line cloud providers, so leak of security is certain. On disks and tapes, then a quick search will find info, if it ever was really secure to begin with.
The point it we need to secure data physically and logically, logical security on its own is at best a hassle.
In later parts of this series, when I get some time. I will explain again self authentication, the vital third part of the network after self encryption and an autonomous secure network. This is where your data maps are made available to you in a pretty fascinating structure. Next week though, I will continue the series on the network itself, that is a very large body of work with a lot of innovation (I believe).
Further details on self encryption can be found here
http://maidsafe.net/SystemDocs/system_components/self_encryption.html
and of course the code is all on github
https://github.com/maidsafe/MaidSafe-Encrypt
Enthusiastic human :-)
Metaquestions 
Great to the point post and very helpful and clear animation (apart I hate to say :-\, from the accent).
The point about physical security is also very well made today with the news that users who had their precious backups deleted without warning by Livedrive, are taking legal action against the company for compensation.
Any centralisation or dependence on the actions of some fallible entity, particularly a business, is to me an unnecessary data security risk. How many times have we seen users left high and dry by system failures, corporate change of heart, bankruptcy or takeover.
I’m looking at Dropbox, Google, Microsoft, Apple etc and thinking: who on earth would trust important data to be preserved, protected and available with any of these guys. Companies always seem like they’ll be around forever when we make such decisions, but very few are, and those that do remain, change. Users are only served so long as they’re profitable.
Thanks Mark, yes this is key. Its more than encrypt and store, that is next to useless and just attracts unwanted attention in todays Internet. Physical security is a major differentiator for SAFE that folks miss out a lot. We will keep shouting though, it’s important 🙂
I liked Shona’s accent and it’s completely understandable to the common man. People are so used to having their privacy compromised that getting their head around the fact that they can actually have this is part of challenge of helping folks to understand that it’s not only possible for MaidSafe are doing it…revolutionary stuff. Yes, keep shouting, David!
In the video, the physical distribution of the chunks is done on the ‘orange’ chunks rather than the ‘purple’ ones, which have gone through the extra XOR process. Is that correct? In addition, what is the purpose of the XORing process, post-encryption?
If you take AES output is should be random, but reversible with a break in the algorithm. If it is xored with one time use pad then its no longer reversible. To break it you need to print all the possible values and GUESS which one it could be and its no easy guess as the result is still encrypted 🙂