Yes, we are the software developers not the medical detectives, but this is a mysterious story of a project that suffered multiple hidden injuries, paralysis attack, comma and eventually death. It was not until the death of the project that the causes of the injuries and paralysis were investigated. The post-mortem analysis gave us an insight of how a project could die. The revelations were shocking and saddening to say the least. Who was the culprit? Who murdered our project? What do we learn from these reports? To know continue reading…
Two years ago there was no shortage of enthusiasm when a sweet little mobile project kicked off. It was a project for one of the US’s Fire Department (details confidential). Handheld devices were all flown from the US to India. The project began with a development of a web service that provided an API for the handheld Windows Mobile devices to retrieve and submit patient data right from the disaster site to the hospital servers. The paramedics would input the patient data on the field and the bits and bytes would flow to the central servers with whatever means was available at the place at that moment. The software was responsible for detecting which means was available using the capabilities of the device. It could be Wi-Fi, GPRS or even the slower Satellite.
The senior members of the team decided to implement this project in .NET compact framework. Initially I was skeptical but the thought of using core C++ libraries instead of the friendlier .NET compact framework scared me and I kept mum.
The application started with simple data entry forms which were rendered on the fly using .NET’s reflection capabilities. The fact that the forms were rendered on the fly by reading an XML file itched me a bit but again it was time to keep mum because this decision was taken by the US guys and not the Indian architects or developers. I continued to keep mum. As the time progressed through the development and testing cycles, more features were added to the application as the requests came in from the clients. The on-device camera was integrated into the application to take snaps of the patient. The bar-code reader was exploited to scan the bar codes on the patient’s wrists. The project was turning into a software adventure.
The .NET Compact Framework got us started stunningly quickly and the initial features were pretty much ready within few months after starting. Thank you .NET Compact Framework. You filled joy in the hearts of our developers. You are so cool. But deep underground below the compact framework and above the Windows CE – the native mobile operating system from Microsoft- was hatching a deadly conspiracy which would eventually threaten and then actually kill this project. I was becoming aware of it but the team did not care as long as the client was not unhappy.
Then came the grand Stress Testing cycles in which testers would enter data of 100s of patients to test the application. The first signs of internal injuries were becoming apparent. The optimization efforts began. Initial investigation quickly revealed memory leaks in the application. Technically speaking they were not memory leaks but the effect of it was similar to memory leaks. In fact, what was shocking to us was we have to think about the usage of the memory now while the framework was supposed to take the burden of the memory management off the shoulders of the developer on to its Garbage Collector. The promise now seems to be a false one. A promise made to lure developers to use the framework. Moreover, the initial success of the project makes the developer think good about the framework. More investigations revealed that the fault was that of the developer, the developers did not properly dispose the objects. What the hell? It’s a programmers’ fault. I could not believe 100% but I knew the framework has played a game with us. The framework promotes bad programming practices or allows sloppy programming which a C++ environment could not have tolerated right from the beginning. We were scared now even to allocate even one new string object. The application was internally bleeding from every side.
Now we knew we needed more memory instantly. Yes the keyword is “instantly”. We wanted to free the memory immediately after use. But the framework did not allow us to do so. Its Disposable protocol meant that the memory will be freed when it thinks it is appropriate. The garbage collector produced spikes in CPU and memory usage when it would run to free memory which we had disposed. By design we did not have control over memory allocations – to be precise – the timing of the memory allocations. Remember we were not building a time critical application; it was just a batch mode application which was not needed to work in real time in the truest sense of the term.
A great application was paralyzed. We could not go back and implement in C++ now. A large chunk of budget was already spent and we were through the development for 2 years now.
Summary: Frameworks are not bad but they produce an illusion of relieving a programmer from the underlying complexity but this complexity attacks back with a vengeance later when the project is in the final stages. A framework like .NET compact framework and some others do have some semi-official documentation available to fix such situations. But this whole thing is really a catch. The frameworks attract you to use it and bites terribly to the extent that project may be rendered unusable. Why not work with the reality right from the beginning rather than start with hiding something as important as memory management? If you do so the benefit is yours in the long run (not very long run). Framework is a silent killer.
WHAT TO DO NOW?