About source code reviewing..
Source code reviewing is the act of reviewing source code (what a definition!) For that to happen, a consultant needs to have access to the source code of the application/framework/module in question. This can be performed automatically, semi-automatically and/or manually. Automatic and semi automatic reviewing can take care of speeding up the review process and can also be focused on identifying flaws that may not be as evident to the human eye. However, products that perform automatic/semi-automatic reviews are not perfect and at least in the year 2007 they are not intelligent; and even though the human mind is not perfect either, products can’t compete with it. Reviewing code manually takes more time than running a beautiful product, yeah, but skilled consultants can identify logic-related problems that would go against any pattern-matching product. So I just mentioned skilled consultants, and the reason behind I wrote that ’skilled’ word is that the quality of the deliverables you get from a manual source code inspection is equal to the skills of the person/people performing the review.I started that paragraph stating that ..consultants need access to.. and it’s also worth mentioning that even though a security consultant would indeed need to have access to the source code in order to perform a manual inspection, it isn’t always the case in other contexts. For instance, inspecting source code during a development process is a common practice these days. It will always be cheaper for a development company to get a product that they can run anytime they have a new code base against the source code, instead of paying someone to do it for them. In that case, they can either have the development team run it (it will depend on how joyful programmers find their new responsibility) or even the QA team. If it’s the QA team, then QA could use the product remotely without having actual access to the source code, reporting any findings back to the development team. So yeah, for a proper source code review a consultant needs to have access to the source code in question, but that isn’t the case at all times in all contexts, which makes sense.
Another thing about source code reviewing is how important it is for a consultant to be able to either compile/run the code or have an environment running the modules under review, with a version as exact as possible (having the exact same source drop isn’t always possible) - Here is when it gets a little dizzy. A comprehensive security review is not always possible and it isn’t just a matter of reviewing source code, so certain limits need to be established for the scope of work. The fact of the matter is that by simply reviewing source code you can have plenty of results but none of those results will turn into actual risks unless you put them in context and test how reliable/exploitable/useful they are. So in order to avoid having a lot of false positives it is always good to have an environment to refer to every now and then during the assessment.
The running environment isn’t just about discarding false positives though - it gives a clearer picture to the person reviewing the source code of what’s going on. Believe me, everyone loves object-oriented programming and abstract paradigms. Wait until you get to review a large-size project simply from a source code perspective and let me know how much time it would take you to trace down the interoperability between each module. I’m not saying it’s impossible, no, but believe me how much an environment would help in there.
But then again, I think I read once in Mark Curphey’s blog about the importance of defining a good statement of work, proposal, scope of work - and it’s absolutely true. It will get very confusing for a client to understand why you need to have an environment if you’re saying you will only perform a manual source code inspection. Of course it’s better for them if you end up testing more than what they thought they were paying for but forget about not having headaches while trying to justify any further ballpark figures.