The need of periodic technology research

Notice: Originally this text was prepared as research proposal over half year ago. Because the company hasn't accepted it, I'm putting on blog a revisited version (generalized). Maybe it will help somebody to convince his company to agree on this kind of technology evaluation method. I also want to thank in this way the bloggers, because I much later used several other people blogs searching materials for motivating developers.

Information technology is a discipline that changes rapidly. Very often new technologies emerge that provide faster application development or cut of costs. Almost all IT professionals agree that new programming languages or frameworks published every new year may invalidate our technology or solution decisions very fast. Something that was the best solution just 6 months ago may not be now!

Availability of many frameworks (especially web frameworks) "pushes" us to compare our own solution to solutions provided by others. Are our algorithms really best possible? Maybe some things can be done faster? Maybe some frameworks offer more functionality than our but we are have more elasticity? Maybe other systems or languages are more scalable or simply faster? Maybe they offer something we don't because we can't even imagine something like that? Maybe our own solutions are the best, fastest & most secure? There will always be a maybe if we don't check newly released alternatives from time to time.

Frameworks

I think many Web programmers want to create own framework. I created big part of one and it was real fun (you know, design from scratch, coding your own vision etc.). But creating also mean maintaining it, looking for improvements, seeking common parts, because creating good framework in the very first release is very, very hard. I know that the others solutions may not have exactly what you want, but there are many reasons why I suggest gaining information about other frameworks:

• Even if you do not use any other tested frameworks, you get knowledge about alternative solutions for the same problems. Effect: you find optimal solution for new problem faster, lowering possible risks and cutting costs of implementing it (not mentioning you do not need to reinvent the wheel).
• If you know that some framework offers almost ready solution or is a better starting point than our own framework for one concrete task and not the other, you may recommend it and possibly lower cost of new project creation by a factor of 2 to 5 and also cut maintenance cost. Without knowledge of alternative frameworks you may miss opportunity to implement something with much less effort. It can turn out that the knowledge of "shortcut" solution makes a difference between success and failure of your project.
• Even if someone does not want to use other framework as the whole, may see profit in using one of its components, because of its maturity or easy of use. In-house tools are good because we know them well and can change them easily, but from product cost perspective it is not always the best solution. For example, Hibernate (object-relational mapping subsystem) for Java is available as open source, is good tested, fast etc. Cost of implementation our own system of that size is hundreds of thousands of Euros (not to mention time needed to implement it). ROI when using existing open source library is much faster and if it is used by many companies and there is rather small possibility that it disappear in near future (if it will happen, you have the code, so it is not so bad). Cost of maintenance is also smaller, because not everything is on our head.
• Open source frameworks sometimes have big communities that create plug-ins etc. A good example is JavaScript jQuery framework. Framework is useful but community created plug-ins are giving it even more power (for example: form validation, user interface blocking, auto completion). Those plug-ins can greatly cut work needed to get a complicated feature done and can even be the only reason to use main framework.

Mentioned reasons are important not only when you start new project but also when you already have some in-house framework. Of course for a starting company or totally new product area there is many more reasons to look for and use frameworks.

Programming languages

Are you sure that the language you are using serves you well? Maybe you think that it serves you only because you didn't test alternatives? Every language is changing but also the other languages. Some companies use several core languages; some choose language per project basis; some stick to one and no other because of maintenance & support reasons. If you are glued to only one technology, is it really the best, fastest (if you care about it) and most productive (may be the main reason) tool for all Web development? For now, you probably cannot tell and can only speculate. Maybe Y or Z give you performance boost so you can use only half of servers for the same traffic. If development time when using them not increases (in long term), maybe there should be room for them. The same goes to productivity: 20% drop of speed may be a good choice if productivity will be doubled.

Programmer and sponsor perspective

This part summaries advantages and disadvantages of periodic technology research from divided by developer and sponsor perspective.

There are two main goals:

• programmer's goal - to get knowledge about new Web creation technologies, learn new algorithms & design strategies;
• sponsor's goal - to get better knowledge of alternative systems that can get new projects done faster, cheaper or with more features or reliable.

Advantages

Developer perspective

From developer perspective learning new way of create software has many advantages:

• It is like a course, because give possibility to learn new things and use it in future work.
• He can see and learn different solutions to similar problems, so in future projects make algorithms faster or designs better (and possible create them with less effort).
• Create better estimates and technology proposition for new projects because of more knowledge of alternatives.
• Get fun testing something new and charge creativity batteries.

Sponsor & HR perspective

From sponsor perspective learning new way of create software also has advantages:

• He knows more ways to create the same thing, so he can do a better (more educated) choice in new project research and design phases.
• Programmers know more software disciplines and it is always a good thing even if there is not an obvious connection of research topic with next projects (they can use some very good design patterns in in-house tools and cut development cost or get more elasticity).
• Many times the fastest way to do work is to not do it (meaning use something that already exists). If exists almost ready solution that one only need to tweak, maybe the one should use it. Without research he may miss the easy way.
• Cost of the research is not much bigger than cost of sending each person to one week course, but he get some practical knowledge and sponsor gets some comparison data.

Disadvantages

I see only one main disadvantage. There is not an easy ROI value for the research, so you spend money but do not easily see profits from doing it. It is a big probability that it may shorten creation of future projects but it is not 100% certain and it is hard to measure. Also not all future projects will profit from the research because not all of them can use Web frameworks or other tested tools. I also see smaller disadvantages. You spent money now in one bigger packet and not per new project basis (in their research or design phase). I see this as small disadvantage because after done general research future per project research can be cheaper because programmer already have some knowledge about alternatives. Another disadvantage may be a possible language change in future projects. Change in language must have be very good motivated if it is not a whole company turnover to new language because some programmers may need to learn new language (but for dynamic languages plus Web frameworks use and when programmer know another dynamic language (for example Python), learning curve is really fast; it is much harder to learn someone programming).

Final results

Sponsor after the research will get final report with below information:

• Pros and cons of use Web frameworks or other tools in future projects (for example, framework X is good at automatic creation of administrative interfaces but is generally slow in rendering them; framework Y is very flexible but hard in learning or maintenance).
• Some general pros and cons of programming languages used by above projects (for example, framework X uses special features of language Z, it halves code needed and give good performance boost).
• Detailed time & hardness of implementation data for example application in compared frameworks. It should also include performance, lines of code measures, pointed out things done differently in this framework (with detailed cause). If some company uses in-house framework, it should be included in comparison.
• Conclusions with programmer's perspective recommendations for future projects (several recommendations depending on project characteristic).

I think sponsor should also consider doing smaller versions of the research every half year, possibly with other group of programmers.