Wednesday, May 25, 2011
Tuesday, May 10, 2011
Backing up family home video
Use Case
Tsunamis, earthquakes, floods, fire, tornadoes, and just plain carelessness threaten to destroy our data. Like most “Gen Y” fathers, it’s important to me to preserve family memories digitally to preserve the quality, and though it’s unnerving to hear reports about the short lifespan of our digital media storage media (especially DVDs), I make backups and no two copies are likely to fail in exactly the same way.
My digital memories are stored on commercial-grade hard drives with mirrored RAID, and as soon as I get a SMART alert I start replacing the drives. That’s my first line of defense. Preserving this data is high value for me, and I’m willing to pay more. However, most natural disasters will destroy any two hard drives in the same computer, so to be truly safe, I need to store my files offsite. Given that I do have local copies, I don’t need enterprise grade offsite backup – (if the backup service reports that files are lost, I’ll be annoyed but I will still have another copy and can upload them again).
My digital HD video camera produces high quality video, but the files are large – often over 4GB (which I realize poses a problem for 32-bit applications in that the integer of the file size is over 32 bits!). A family vacation can easily be 10GB. Photos from a birthday party can easily be 50 megabytes with a standard 5 or 8 megapixel digital camera, more if friends share their photos with me. If I have to pay per megabyte, this storage gets expensive quickly. It also becomes time-consuming to back up, so I really only want to do it once. The cost of switching services is high.
In my attempts to address this use case, I tried several backup services to accomplish the job.
Carbonite
Granted this review is a bit dated, but when I started using Carbonite in about 2009, the service worked well, but it was hard to tell.
Technology
Carbonite compresses and encrypts the file before transmission, providing security for sensitive documents like Quicken and that password list I share with my wife (that’s probably another missing use case that deserves another post).
Home videos in the gigabyte range backup very slowly, so not having a progress bar made it hard to tell what was going on, and Carbonite tries not to tax the system while it’s in use, meaning it was hard to tell when it was actively sending data to the data center (every time I checked on it, the computer was in use). Carbonite also used a lot of processing power even when it wasn’t transmitting, which I realize contradicts what I just said about not taxing the system, but that was a year or two ago and my memory of that time isn’t all that sharp.
The backup took about 3 months to complete (about 80GB from what I remember). At first, the backup would stop and start over again (? – never could get a straight answer from Support on whether that’s what was going on) if the machine suspended, so I left it running all the time. It increased my electricity bill substantially but I got a lot of BOINC work units done in the process.
My switch from Windows XP to Windows 7 was a bit rocky, but I got through it. The experience with switching to Windows 7 is what drove me to consider alternatives.
Loyalty and Referral Program
The business model allowed me to get additional months for free if I recommended my friends, but Carbonite is bundle with every new computer out there and well-advertised, so I never got bonus months out of it. I sent my friends links to the software, and they found they already had it installed on their new PCs, or they Googled it and found it quite easily. Finding my email again and clicking the link was higher effort than using the software they already had, or they didn’t really have the need to backup. Good tactic, but in reality, not much value for the referrer, and the new customer gets the same deal that they would get clicking on the icon on their HP notebook’s desktop.
Support
Though I felt the support team was giving me canned answers most of the time and I questioned their response to me that the bottleneck was on my end (“Try deselecting files in your backup” “Don’t backup videos” “Try not backing up files over 4GB”), they wereresponsive at least. …and I understand that large files take time, so yes, I could try not backing up videos, but that’s probably the most important thing I want to preserve!
Mozy
Technology
Mozy was the best. The software worked well, it was affordable, I didn’t need to contact support at all – it just worked. Best of all, I could easily see the backup progress. Yes, It took forever to send the large files, but I could backup what I needed to backup. The software was invisible.
It was so invisible that I forgot I had it installed until late February, when it informed me that the Unlimited data plan was being discontinued. I would have been willing to pay more for Mozy to keep the unlimited plan and not have to start over with a new service, but Mozy was apparently no longer interested in my business.
Loyalty and Referral Program
I think they had a loyalty program comparable to Carbonite’s, but I’m not really sure. Again, the service was so invisible that I forgot it was even there. When friends and relatives asked me about backup services, I told them about Mozy of course. I was referring them even without the benefits of the referral program.
Support
I chose Mozy because others had reported the support was excellent and the product was great. I never needed to use Mozy’s technical support because it was so fantastic.
When Mozy decided to end the unlimited plan, they gave me a month to decrease the data I had stored down to 50 GB, but now having over twice that in family pictures and video, that was an unreasonable ask. After freezing my account, they gave me another month before deleting my data.
For people with lesser needs, I’ll certainly recommend Mozy, but they no longer meet my needs.
LiveDrive
In one line or less: LiveDrive is not suitable, and borderline scam quality. The configuration settings don't work, support response time is lengthy, and they don’t honor their trial policy. Read on for the whole story.
I’d had such a tough time with Carbonite’s technology that “the devil you know is better than the devil you don’t know” just didn’t hold for me, so I looked for alternatives. LiveDrive was available through Upromise.com and through CompUSA.com, so I thought it would be worth trying. The price was reasonable, they had an unlimited data plan, and they were offering a 45-day trial.
During the signup process, (and I have to give their Sales team some credit for thinking of this), they offered me the opportunity to upgrade to a prepaid 3 year plan with substantial discounts. I would still have 30 days to cancel. I figured that probably would be enough time to decide whether or not the service would work out for me, and again, I hate switching because it takes so long to do the initial backup, so whatever I choose I'll stick with for a long time.
Technology and Lack of support
As I mentioned before, the highest priority backups are HD home videos of my family, which are quite large. So it took me about 3 weeks to realize that it was backing up, yes, some of my multi-gigabyte home movies, but also a lot of other movies that I had purposely deselected and excluded. These are videos that are available from a subscription service that can always be downloaded again later, so they just take up precious time and electricity to back it up.
LiveDrive was also continuously distracted by BOINC. BOINC is constantly changing files because it’s working on new work units with my spare processor power, solving world hunger and finding cures for cancer. …and LiveDrive felt the need to back up all of those files as they changed, even though the directory had been excluded.
Not only that, but it was backing the same files over and over again (even the ones that weren’t changing). I tried changing some settings to exclude those directories, but that didn’t change LiveDrive’s behavior. With a few days to spare of the 30 day trial (and again, not really wanting to change and figuring we could work this out), I contacted LiveDrive support by email, and said if we couldn't figure this out, I would have to cancel.
After a couple of days I called them in the UK, pressed “1” for support, and immediately got disconnected. Thinking it was a fluke, I tried a few more times, same results.
I let it go past the end of the trial because this is such an obvious problem – no one could possibly build a business around this when the software keeps starting over and backing up the same files, right? …and I really don’t want to switch - it's already backed up 20 Gigs or so that I don't want to have to backup again.
After two weeks went by with no response from the support team, I called my credit card company and asked them to help me get a refund. I went to Vark.com to ask someone to recommend an alternative, and started backing things up with a new service.
One month after my original support request, I finally got a response from support and was informed that they would not be issuing a refund because I’d gone past the trial period. I tried to negotiate with them and get at least a portion of the $160 I’d spent for 3 years, and they refused. They don’t offer telephone technical support, and if I wanted to come back and troubleshoot some more, they would do that, but I would not be getting my money back, or even a portion of that. It’s against policy. Yes, they acknowledged I did contact them about the problem within the trial period, but 2 days isn’t reasonable turnaround time, so they won’t honor the terms of the trial.
I’m a businessman. I understand why web-based software services built on the SaaS model are reluctant to provide refunds. I acknowledge that I didn’t give them a week to fix the problem, and I acknowledge that I knew they had policies against providing refunds outside the trial period.
However I also understand the value of happy customers, and more importantly, the threat of angry customers. If I were in their shoes, I would offer to refund all but 1 month or even 1 year. I would attempt to grow the customer base through word of mouth like Carbonite was doing.
This seems to be the opposite model. Ignore the customer for as long as possible, wait for them to try something else, don't offer to negotiate, and make them just hopping mad. Well, if that was the goal, it worked. I am hoping my credit card company can do something to help recover a portion of this.
I very strongly recommend against using this service.
BackBlaze
Someone on Vark.com / Aardvark recommended this service to me. Like the other services, by default it wants to backup everything, and offers exclusion by file extension and/or by directory. Though I feel BackBlaze is not making nearly as much progress as Mozy did in the same time in terms of gigabytes backed up, the service can be interrupted, doesn’t backup the same files over and over again, honors its exclusion lists, and my DSL modem is constantly blinking. Like Mozy, it doesn’t tax the system and I forget it’s even there.
I’m reluctant to give it an A just yet because I haven’t seen how they try to resolve customer problems and I reserve A’s for the truly exceptional services, but so far so good.
Symantec
I work for Symantec (note: the opinions stated here are solely my own and do not represent the views of Symantec), and feel I should give an obligatory nod to Symantec’s own consumer online backup service. Norton Online Backup does offer storage space for an annual fee, but its limitation of 25GB makes it unsuitable for this use case.
I may consider them if they start offering mobile backup services.
At work, I use Backup Exec to keep a spare copy of my entire hard drive. This is similar to my RAID solution at home, and works great for a laptop solution, though not ideal if an earthquake strikes while I'm in the office.
Adobe
Adobe also has a backup solution that works with Photoshop Elements to backup pictures, but it's not really designed for large home videos, and it's much more expensive.
Mobile
One thing not mentioned above is that when choosing other services, I was also looking for a service that would let me backup files from my mobile device. MyBackup Pro is my current mobile data backup solution, but there is value in reducing complexity and having just one vendor managing my data. No such vendor currently exists.
Summary
One thing that none of the services can do is to jumpstart that initial backup by being able to e.g. mail a hard drive (which I'd be willing to do just to have an offsite copy). All of the services claim that the initial backup takes a few weeks, but for me it takes a few months.
Tsunamis, earthquakes, floods, fire, tornadoes, and just plain carelessness threaten to destroy our data. Like most “Gen Y” fathers, it’s important to me to preserve family memories digitally to preserve the quality, and though it’s unnerving to hear reports about the short lifespan of our digital media storage media (especially DVDs), I make backups and no two copies are likely to fail in exactly the same way.
My digital memories are stored on commercial-grade hard drives with mirrored RAID, and as soon as I get a SMART alert I start replacing the drives. That’s my first line of defense. Preserving this data is high value for me, and I’m willing to pay more. However, most natural disasters will destroy any two hard drives in the same computer, so to be truly safe, I need to store my files offsite. Given that I do have local copies, I don’t need enterprise grade offsite backup – (if the backup service reports that files are lost, I’ll be annoyed but I will still have another copy and can upload them again).
My digital HD video camera produces high quality video, but the files are large – often over 4GB (which I realize poses a problem for 32-bit applications in that the integer of the file size is over 32 bits!). A family vacation can easily be 10GB. Photos from a birthday party can easily be 50 megabytes with a standard 5 or 8 megapixel digital camera, more if friends share their photos with me. If I have to pay per megabyte, this storage gets expensive quickly. It also becomes time-consuming to back up, so I really only want to do it once. The cost of switching services is high.
In my attempts to address this use case, I tried several backup services to accomplish the job.
Service | Grade |
---|---|
Carbonite | C+ |
Mozy | |
LiveDrive | F- |
BackBlaze | B |
Symantec | N/A |
Adobe | N/A |
Carbonite
Granted this review is a bit dated, but when I started using Carbonite in about 2009, the service worked well, but it was hard to tell.
Technology
Carbonite compresses and encrypts the file before transmission, providing security for sensitive documents like Quicken and that password list I share with my wife (that’s probably another missing use case that deserves another post).
Home videos in the gigabyte range backup very slowly, so not having a progress bar made it hard to tell what was going on, and Carbonite tries not to tax the system while it’s in use, meaning it was hard to tell when it was actively sending data to the data center (every time I checked on it, the computer was in use). Carbonite also used a lot of processing power even when it wasn’t transmitting, which I realize contradicts what I just said about not taxing the system, but that was a year or two ago and my memory of that time isn’t all that sharp.
The backup took about 3 months to complete (about 80GB from what I remember). At first, the backup would stop and start over again (? – never could get a straight answer from Support on whether that’s what was going on) if the machine suspended, so I left it running all the time. It increased my electricity bill substantially but I got a lot of BOINC work units done in the process.
My switch from Windows XP to Windows 7 was a bit rocky, but I got through it. The experience with switching to Windows 7 is what drove me to consider alternatives.
Loyalty and Referral Program
The business model allowed me to get additional months for free if I recommended my friends, but Carbonite is bundle with every new computer out there and well-advertised, so I never got bonus months out of it. I sent my friends links to the software, and they found they already had it installed on their new PCs, or they Googled it and found it quite easily. Finding my email again and clicking the link was higher effort than using the software they already had, or they didn’t really have the need to backup. Good tactic, but in reality, not much value for the referrer, and the new customer gets the same deal that they would get clicking on the icon on their HP notebook’s desktop.
Support
Though I felt the support team was giving me canned answers most of the time and I questioned their response to me that the bottleneck was on my end (“Try deselecting files in your backup” “Don’t backup videos” “Try not backing up files over 4GB”), they wereresponsive at least. …and I understand that large files take time, so yes, I could try not backing up videos, but that’s probably the most important thing I want to preserve!
Mozy
Technology
Mozy was the best. The software worked well, it was affordable, I didn’t need to contact support at all – it just worked. Best of all, I could easily see the backup progress. Yes, It took forever to send the large files, but I could backup what I needed to backup. The software was invisible.
It was so invisible that I forgot I had it installed until late February, when it informed me that the Unlimited data plan was being discontinued. I would have been willing to pay more for Mozy to keep the unlimited plan and not have to start over with a new service, but Mozy was apparently no longer interested in my business.
Loyalty and Referral Program
I think they had a loyalty program comparable to Carbonite’s, but I’m not really sure. Again, the service was so invisible that I forgot it was even there. When friends and relatives asked me about backup services, I told them about Mozy of course. I was referring them even without the benefits of the referral program.
Support
I chose Mozy because others had reported the support was excellent and the product was great. I never needed to use Mozy’s technical support because it was so fantastic.
When Mozy decided to end the unlimited plan, they gave me a month to decrease the data I had stored down to 50 GB, but now having over twice that in family pictures and video, that was an unreasonable ask. After freezing my account, they gave me another month before deleting my data.
For people with lesser needs, I’ll certainly recommend Mozy, but they no longer meet my needs.
LiveDrive
In one line or less: LiveDrive is not suitable, and borderline scam quality. The configuration settings don't work, support response time is lengthy, and they don’t honor their trial policy. Read on for the whole story.
I’d had such a tough time with Carbonite’s technology that “the devil you know is better than the devil you don’t know” just didn’t hold for me, so I looked for alternatives. LiveDrive was available through Upromise.com and through CompUSA.com, so I thought it would be worth trying. The price was reasonable, they had an unlimited data plan, and they were offering a 45-day trial.
During the signup process, (and I have to give their Sales team some credit for thinking of this), they offered me the opportunity to upgrade to a prepaid 3 year plan with substantial discounts. I would still have 30 days to cancel. I figured that probably would be enough time to decide whether or not the service would work out for me, and again, I hate switching because it takes so long to do the initial backup, so whatever I choose I'll stick with for a long time.
Technology and Lack of support
As I mentioned before, the highest priority backups are HD home videos of my family, which are quite large. So it took me about 3 weeks to realize that it was backing up, yes, some of my multi-gigabyte home movies, but also a lot of other movies that I had purposely deselected and excluded. These are videos that are available from a subscription service that can always be downloaded again later, so they just take up precious time and electricity to back it up.
LiveDrive was also continuously distracted by BOINC. BOINC is constantly changing files because it’s working on new work units with my spare processor power, solving world hunger and finding cures for cancer. …and LiveDrive felt the need to back up all of those files as they changed, even though the directory had been excluded.
Not only that, but it was backing the same files over and over again (even the ones that weren’t changing). I tried changing some settings to exclude those directories, but that didn’t change LiveDrive’s behavior. With a few days to spare of the 30 day trial (and again, not really wanting to change and figuring we could work this out), I contacted LiveDrive support by email, and said if we couldn't figure this out, I would have to cancel.
After a couple of days I called them in the UK, pressed “1” for support, and immediately got disconnected. Thinking it was a fluke, I tried a few more times, same results.
I let it go past the end of the trial because this is such an obvious problem – no one could possibly build a business around this when the software keeps starting over and backing up the same files, right? …and I really don’t want to switch - it's already backed up 20 Gigs or so that I don't want to have to backup again.
After two weeks went by with no response from the support team, I called my credit card company and asked them to help me get a refund. I went to Vark.com to ask someone to recommend an alternative, and started backing things up with a new service.
One month after my original support request, I finally got a response from support and was informed that they would not be issuing a refund because I’d gone past the trial period. I tried to negotiate with them and get at least a portion of the $160 I’d spent for 3 years, and they refused. They don’t offer telephone technical support, and if I wanted to come back and troubleshoot some more, they would do that, but I would not be getting my money back, or even a portion of that. It’s against policy. Yes, they acknowledged I did contact them about the problem within the trial period, but 2 days isn’t reasonable turnaround time, so they won’t honor the terms of the trial.
I’m a businessman. I understand why web-based software services built on the SaaS model are reluctant to provide refunds. I acknowledge that I didn’t give them a week to fix the problem, and I acknowledge that I knew they had policies against providing refunds outside the trial period.
However I also understand the value of happy customers, and more importantly, the threat of angry customers. If I were in their shoes, I would offer to refund all but 1 month or even 1 year. I would attempt to grow the customer base through word of mouth like Carbonite was doing.
This seems to be the opposite model. Ignore the customer for as long as possible, wait for them to try something else, don't offer to negotiate, and make them just hopping mad. Well, if that was the goal, it worked. I am hoping my credit card company can do something to help recover a portion of this.
I very strongly recommend against using this service.
BackBlaze
Someone on Vark.com / Aardvark recommended this service to me. Like the other services, by default it wants to backup everything, and offers exclusion by file extension and/or by directory. Though I feel BackBlaze is not making nearly as much progress as Mozy did in the same time in terms of gigabytes backed up, the service can be interrupted, doesn’t backup the same files over and over again, honors its exclusion lists, and my DSL modem is constantly blinking. Like Mozy, it doesn’t tax the system and I forget it’s even there.
I’m reluctant to give it an A just yet because I haven’t seen how they try to resolve customer problems and I reserve A’s for the truly exceptional services, but so far so good.
Symantec
I work for Symantec (note: the opinions stated here are solely my own and do not represent the views of Symantec), and feel I should give an obligatory nod to Symantec’s own consumer online backup service. Norton Online Backup does offer storage space for an annual fee, but its limitation of 25GB makes it unsuitable for this use case.
I may consider them if they start offering mobile backup services.
At work, I use Backup Exec to keep a spare copy of my entire hard drive. This is similar to my RAID solution at home, and works great for a laptop solution, though not ideal if an earthquake strikes while I'm in the office.
Adobe
Adobe also has a backup solution that works with Photoshop Elements to backup pictures, but it's not really designed for large home videos, and it's much more expensive.
Mobile
One thing not mentioned above is that when choosing other services, I was also looking for a service that would let me backup files from my mobile device. MyBackup Pro is my current mobile data backup solution, but there is value in reducing complexity and having just one vendor managing my data. No such vendor currently exists.
Summary
One thing that none of the services can do is to jumpstart that initial backup by being able to e.g. mail a hard drive (which I'd be willing to do just to have an offsite copy). All of the services claim that the initial backup takes a few weeks, but for me it takes a few months.
- Carbonite probably has addressed my biggest complaints by now and is worth revisiting.
- Mozy’s unlimited data storage service was excellent, but is now discontinued. They no longer have a solution that meets my needs.
- LiveDrive should be blacklisted.
- BackBlaze seems to be the closest thing to an ideal solution. A little sluggish perhaps, but it works.
- Symantec and Adobe are trusted brand names that could build market share quickly, but don't currently address this use case.
Thursday, April 14, 2011
Features: less is more
Imagine this: You are in the market for a new blender. All you're going to do with it is to make smoothies and milkshakes. If it grinds ice and the parts are dishwasher safe, that will do.
You walk into your favorite small appliance store and see a row of blenders. None of the prices are shown. Almost all of the blender boxes say that they will make smoothies and milkshakes. Two of them say they're dishwasher safe. One of those two blenders can also shred metal, is easy to take apart, has measuring markers on the side, and comes with a set of six exchangeable lids, identical except that they're white, black, green, brown, red, and pink.
But again, it's hard to tell the price. The cash register is downstairs at the far end of the store, and there are no price checking stations to be found anywhere.
Which would you choose? Most people would choose the simpler blender. Why? Fit for the customer's needs, and "I don't need all of those extra things," and "It will cost me more."
In product management, whether it's software, blenders, or even a resume the concept is the same -- less is more. Your product needs to fit the customer's needs, but not beyond that. More is not necessarily better. In fact, the company could be earning a much smaller margin on each blender because of the cost of those additional features and the requirement to keep the price under a certain level.
By the way, I checked the web site - the simple-looking blender actually has many of the features of the fancy one yet still earned the sale. ...but that's more of an outbound marketing issue, isn't it?
You walk into your favorite small appliance store and see a row of blenders. None of the prices are shown. Almost all of the blender boxes say that they will make smoothies and milkshakes. Two of them say they're dishwasher safe. One of those two blenders can also shred metal, is easy to take apart, has measuring markers on the side, and comes with a set of six exchangeable lids, identical except that they're white, black, green, brown, red, and pink.
But again, it's hard to tell the price. The cash register is downstairs at the far end of the store, and there are no price checking stations to be found anywhere.
Which would you choose? Most people would choose the simpler blender. Why? Fit for the customer's needs, and "I don't need all of those extra things," and "It will cost me more."
In product management, whether it's software, blenders, or even a resume the concept is the same -- less is more. Your product needs to fit the customer's needs, but not beyond that. More is not necessarily better. In fact, the company could be earning a much smaller margin on each blender because of the cost of those additional features and the requirement to keep the price under a certain level.
By the way, I checked the web site - the simple-looking blender actually has many of the features of the fancy one yet still earned the sale. ...but that's more of an outbound marketing issue, isn't it?
Thursday, April 7, 2011
Privacy in social media
There's an old story of two hikers who come across a hungry-looking bear in their path. The first hiker starts running for his life. The second hiker stops, sits down on a rock, takes off his heavy hiking boots off, reaches into his backpack and pulls out a pair of running shoes.More on that story in a moment.
The first hiker yells back at him, "What the heck are you doing? You know you can't outrun a bear!" To which the second hiker matter-of-factly responds, "I don't have to outrun the bear, I just have to outrun you."
Whether it's Twitter geolocated tweets, photos that include GPS metadata, or your Facebook profile, you have the option of sharing with the world where you are. If you share pictures of your kids that include GPS coordinates and most of them are clearly taken at home, you've just advertised where that child lives. ...and as you probably already know, there are bad, crazy people out there who are looking for you and your kids for any number of reasons.
Note to self, recheck pictures on Facebook to see if they contain GPS coordinates.
What you may not realize is that you are one entry in a phone book. I'm not saying it's okay to share with the world that you're about to be a mother or father and all baby gifts can reach you at P. Sherman, 42 Wallaby Way, Sydney, Australia, but adding to your Twitter profile that you live in Sydney is probably fine because Sydney is, after all, a pretty big place. So is New York, San Francisco, San Jose, etc. If you had a lot of enemies, could they still find you? Yeah, Zabasearch is one way to do it. For most of us, though, it's a lot like the story of the bear above. You're aiming to not be the easiest target.
Also remember that most people are too busy thinking about themselves to obsess about you, so unless you come across someone particularly crazy and vengeful or have decided to plant $100 bills in your front yard and word is getting around, there's no value in someone travelling miles to seek you out.
Tips:
- For most people living in large cities, it's okay to make your city of residence public. Like opting in to newsletters from your favorite stores, sometimes a little bit of information out there can make good things happen (e.g. an old friend decides to get back in touch when in the area).
- Make private matters known only to people you trust.
- Most crime happens among people who know each other, and chances are you know which of your friends and family you can trust.
- Employers may not be "out to get you" but if you're looking to get a job or keep a job, you probably don't want them to read about certain things you post publicly that might reflect badly on them.
- Facebook and Twitter are by default not private, so even if you decide to keep that the way it is, just remember who your audience is. More than just your Twitter followers have probably read your post about Charlie Sheen, for example, since it's very easy to search for tweets without ever logging in.
- Your video of the kids posted on Youtube is probably fine, so long as you keep the above points in mind.
Yeah, but we can build things that barely stand up!
In college I remember taking a class at UC Davis in which the professor started a lecture one day saying that, "anyone can build a house. Throw enough materials at it, and it will be a suitable home."
Then, with a smile on his face he added, "but engineers can build a house that barely stands up." He said it as if he was proclaiming the cure for world hunger. Someone in the class murmured, "We can probably do better than that..."
That comment, plus a number of quizzacle looks on students' faces prompted him to explain, "We know how to build a strong house with the minimum amount of materials. Anyone can build a house, but usually it will contain a lot of wasted resources or won't be as strong."
That class was a general Engineering course. How does this apply to software engineering? Thinking back further, I remember reading something from Steve Gibson, whom created SpinRite and other apps, claiming that the efficiency of software had become a lost art, and that a program written to accomplish a fairly simple task now consumes a large amount of disk space, RAM, and processing power when it could have been accomplished within a few hundred kilobytes and under 100 megahertz.
Well okay, that's not an exact quote, what he was getting at is that we've thrown a lot of resources at a problem when the solution could be quite simple. Many programs written in Java are built to be interpreted and compiled on-the-fly, increasing application portability at the expense of size and processing power. Software contains interfaces and procedures for crunching data from other programs that the user may never use, and that would only be activated with the right complimenting technology. In the mean time, it consumes disk space and memory. Visual software development tools make it easy to develop a computer program, though the computer's interpretation of what was intended is sometimes ambiguous and opens security holes.
Entire libraries now have to be shipped with a product even when a few of those functions are used by the program. That's probably not such a big deal -- it's akin to leaving branches on the tree logs used to build your log cabin. A little unsightly, but it's not doing any harm, and there's nothing wrong with the part of the tree that's being used, right? Wrong - if some of the unused libraries are insecure and leverage the parts of the library in use, it would be like putting termites into those branches. Eventually they could reach the core of the house.
This is getting abstract, so let's come back to a real life example of software engineering sloppiness: A few years ago, Magellan sent out an update for its GPS devices that required the user to enter an activation code based on the serial number. The installer interpreted the serial number string as a number. The activation code generator interpreted the serial number as a text field. That wouldn't matter except that since the integer variables didn't bother to store the number of leading 0's in the serial number field, the activation code came out differently depending on whether the leading 0's were read or not.
Here's a security example: Stuxnet used a number of small holes and hooks to get where it needed to go. (http://www.isssource.com/tag/how-stuxnet-spread/) That included USB drives, print spooler services, autorun.inf, Siemens project files, and finally programmable logic controllers.
Put this way, it looks like as we make it easier to build software, we're all following the "anyone can do it" methodology. Software engineering becomes cheap. So going back to the beginning of this article, where's the engineering element of building it right? How do we build strong software with minimal resources? Should we all go back to building software with assembly?
Then, with a smile on his face he added, "but engineers can build a house that barely stands up." He said it as if he was proclaiming the cure for world hunger. Someone in the class murmured, "We can probably do better than that..."
That comment, plus a number of quizzacle looks on students' faces prompted him to explain, "We know how to build a strong house with the minimum amount of materials. Anyone can build a house, but usually it will contain a lot of wasted resources or won't be as strong."
That class was a general Engineering course. How does this apply to software engineering? Thinking back further, I remember reading something from Steve Gibson, whom created SpinRite and other apps, claiming that the efficiency of software had become a lost art, and that a program written to accomplish a fairly simple task now consumes a large amount of disk space, RAM, and processing power when it could have been accomplished within a few hundred kilobytes and under 100 megahertz.
Well okay, that's not an exact quote, what he was getting at is that we've thrown a lot of resources at a problem when the solution could be quite simple. Many programs written in Java are built to be interpreted and compiled on-the-fly, increasing application portability at the expense of size and processing power. Software contains interfaces and procedures for crunching data from other programs that the user may never use, and that would only be activated with the right complimenting technology. In the mean time, it consumes disk space and memory. Visual software development tools make it easy to develop a computer program, though the computer's interpretation of what was intended is sometimes ambiguous and opens security holes.
Entire libraries now have to be shipped with a product even when a few of those functions are used by the program. That's probably not such a big deal -- it's akin to leaving branches on the tree logs used to build your log cabin. A little unsightly, but it's not doing any harm, and there's nothing wrong with the part of the tree that's being used, right? Wrong - if some of the unused libraries are insecure and leverage the parts of the library in use, it would be like putting termites into those branches. Eventually they could reach the core of the house.
This is getting abstract, so let's come back to a real life example of software engineering sloppiness: A few years ago, Magellan sent out an update for its GPS devices that required the user to enter an activation code based on the serial number. The installer interpreted the serial number string as a number. The activation code generator interpreted the serial number as a text field. That wouldn't matter except that since the integer variables didn't bother to store the number of leading 0's in the serial number field, the activation code came out differently depending on whether the leading 0's were read or not.
Here's a security example: Stuxnet used a number of small holes and hooks to get where it needed to go. (http://www.isssource.com/tag/how-stuxnet-spread/) That included USB drives, print spooler services, autorun.inf, Siemens project files, and finally programmable logic controllers.
Put this way, it looks like as we make it easier to build software, we're all following the "anyone can do it" methodology. Software engineering becomes cheap. So going back to the beginning of this article, where's the engineering element of building it right? How do we build strong software with minimal resources? Should we all go back to building software with assembly?
Sunday, August 29, 2010
Monday, August 23, 2010
The Customer's Train
The customer told the product manager he needed a new train to go from New York to San Francisco.
The product manager commissioned his engineers to build it.
It would cost one million dollars.
So the money was spent.
The product manager found the train inadequate. "It needs to carry passengers," he said.
It would cost another three million dollars.
So the money was spent.
The product manager found the train inadequate. "We need to feed passengers on this journey," he said.
It would cost another six million dollars.
So the money was spent.
The product manager found the train inadequate. "Surely the passengers will need to carry luggage too," he said.
It would cost another three million dollars.
So the money was spent.
The product manager found the train inadequate. "The train should be efficient -- build me a train that uses less fuel," he said.
It would cost another eight million dollars.
So the money was spent.
The customer called. He said a competitor had finished building the train he wanted.
It would cost less than ten thousand dollars.
So the product was bought.
The competitor was Brio.
The buyer worked for Amtrak.
The user was his 8-year-old son.
New York and San Francisco were models the buyer had built for his son.
Always remember the difference between the buyer and the user.
Build according to the user's needs.
The product manager commissioned his engineers to build it.
It would cost one million dollars.
So the money was spent.
The product manager found the train inadequate. "It needs to carry passengers," he said.
It would cost another three million dollars.
So the money was spent.
The product manager found the train inadequate. "We need to feed passengers on this journey," he said.
It would cost another six million dollars.
So the money was spent.
The product manager found the train inadequate. "Surely the passengers will need to carry luggage too," he said.
It would cost another three million dollars.
So the money was spent.
The product manager found the train inadequate. "The train should be efficient -- build me a train that uses less fuel," he said.
It would cost another eight million dollars.
So the money was spent.
The customer called. He said a competitor had finished building the train he wanted.
It would cost less than ten thousand dollars.
So the product was bought.
The competitor was Brio.
The buyer worked for Amtrak.
The user was his 8-year-old son.
New York and San Francisco were models the buyer had built for his son.
Always remember the difference between the buyer and the user.
Build according to the user's needs.
Subscribe to:
Posts (Atom)