I wrote this article after helping someone who had to engage a product development company for the first time. It's not really for people who work in product development professionally, or for larger projects. Nevertheless, it does contain though a lot of things I wish I'd known earlier in my own career when comparing product development partners.

Introduction at some point you may have a software product idea that you need to turn into reality. This can be both an exciting and a scary time. Often, you won’t have the development skills to build a product yourself so you’ll almost inevitably need to engage an agency or software development company to help. It’s a big step though to take your carefully nurtured idea and release it into someone else’s hands. This relationship can end up fractious and acrimonious even at the best of times, so choosing the best partner is incredibly important. I should note that this guide is aimed at people with little or no software product experience, taking on projects between £10K and about £200K. If they get much larger, you really should consider engaging professional expertise.

Right now, I work with both sides of this process: as a buyer in my role as CIO of SmartDebit and as supplier in my non-exec director role at Open Frequency. However, I've experienced both sides of this divide for many years, so I thought it might be helpful to note some things I wish I'd known earlier in my career.

I should point out here that it doesn’t matter whether it’s an app, a website or an enterprise software system, or whether you represent a company or an individual; the same concerns apply when choosing a partner. You will also have to be aware that for no sinister reasons, not all suppliers will suit all clients – their working processes just have to click with yours.

One often overlooked way to smooth this whole process is to hire in your own specialist product manager or project manager, separate from the product development company. This can look like a needless expense, but I've saved people significant amounts of time and money with just a few days consultancy. It really helps to have someone who has walked the walk on your side.

If you're going to do this yourself though, these are some areas that I've found are of particular interest: the company itself, its skills and ability to deliver, common gotchas and contractual considerations. There's far more of course that could be said, but these form a good basis for comparison.

The companyAn important thing to consider with any agency is its history and track record. Apart from the obvious checks that it's still going to be here tomorrow, you need to look at how customers are treated. Make sure you verify the following:

Repeat businessHappy customers come back. Dissatisfied ones don’t. Ask them how many of their customers come back, and crosscheck this with their promotional material.

Credible larger clientsThis isn’t because larger clients are better; it’s because they show that the business is real and has been able to attract, engage and deliver to a company that is likely to be bigger, more experienced and difficult to please than an individual or small company. Smaller agencies won't always have this, but larger clients have no issue using small agencies for work, so this evidence may well be there.

Available referencesIt goes without saying that if they have happy clients, those clients will be willing to talk to you. Even if you don’t want to talk to them, get the names and contact details anyway so you know they exist.

Product historyThis is an odd one, but important if you’re taking a product to market. You want to verify how much work the company has done producing real products, rather than one-off campaigns. Campaign apps are not products. Many agencies carry out a lot of “fire and forget” deliveries for single events. It can be their sole service. These don’t need the same kind of quality, scalability and processes in place to be delivered. You need to make sure that the agency (and the individuals within it) have experience delivering products with a lifetime longer than a few months. If they don’t get this right, it’s like trying to build a house on sand when you come to extend and scale your product.

How does it feel meeting them?It might seem an obvious question, but do you get on with them? It's not just about money. They have to feel right to fit with your way of working (or vice versa). In a relationship that will be tested when things inevitably start to fray, you need to know that your working relationship will be solid. Are they reasonable? Are they responsive? Do they get frustrated easily? Do they have the same ethos as you?
Skills and ability to deliverYou’ve found a supplier you trust who has the right track record. You now need to check that they have the right skills to deliver your particular product end-to-end. Just remember that a chain is only as good as its weakest link so if they fail in one of these areas, it may affect the whole project.

Business analysisThe business analysis stage is where you take your ideas and turn them into detailed requirements, like a low-level set of schematics. It’s often helpful to have an independent consultant help with this so you know fully what you want before going to a supplier. That way things don’t get missed. If you do leave it to the agency, make sure you find out if they will estimate costs before scoping or whether there’s a significant outlay first before they will produce an estimate or quote. Have a look at the kind of information they’ll produce during the business analysis phase. If it is comprehensive you can at least make a decision whether to go somewhere else for the build if the quote is too high.

User ExperienceUser experience (UX) is the modern term that includes user interface design. Sometimes it includes actual drawing and designing graphics for the interface but it can also just mean designing how the product will flow, what should be on each screen and how it should be laid out. Have a look at the supplier’s previous projects to see if the user flow is good and if it looks clean and modern. Many people have trouble recognising good design so if you have a design-professional friend, get them involved too.

GraphicsIf the supplier provides graphic design services separately from its products and UX, have a look at their other design materials too. Their designers will do the products as well so you can get a good idea of their product results from any design work they’ve done. If they don’t have designers, find out how they do graphics. If they use external people you may get charged cost rates for any extra work done during development rather than having the flexibility of in-house staff.

If you like their website, ask them if they developed it themselves. I’ve seen seen several agencies that get someone else to build their site when it’s a service they claim to offer expertise in themselves. If on the other hand you don’t like their website, why are you considering them as an agency?

DevelopmentLook at the pedigree of the supplier’s programmers. What have they done? Anything recognisable or big? How much experience do they have in the technologies needed for your project? Programming skills are probably the hardest thing to assess in product development, so you’ll need to rely on CVs, experience level and previous projects.

If you have engaged your own independent specialist, he or she can also vet the supplier for skills that would be relevant to the way he or she thinks your project should be built, or even help negotiate that approach with the supplier.

TestingHow does the supplier test things? Do they employ specialist (and ISEB or equivalently qualified) testers? What is their test regimen? Do they have automated testing as well as manual testing? What are considered passing and failing bugs (this is important to define at contract as something that may be really important to you may just be trivial or below the fix threshold for a supplier - check how they define acceptable issues)?

Early live supportWhat’s their warranty once you go live? How long is it valid for? What is out of scope? What's on paper may look good, but here’s where talking to previous clients can really help.

Ongoing relationshipWhat happens once your product is built, deployed and being used? Give some thought to how you will work with the agency thereafter, and whether you need specific terms in the contract to handle this. If you will be hiring your own team, make sure the handover costs are defined and that any essential documentation is included. If you want to continue the relationship with the agency, make sure you have a good idea of the costs and rates for follow-up work.

Company sizeIt doesn't really matter how big the supplier is. What matters is what resources can they draw upon when things go wrong. What happens if their key programmer is ill? Who is going on holiday during your project? What happens if a much more important client needs something done in the middle of your project? It's worth investigating the back-up resources available to you. For a small company, there may be nothing, but at least you can tell if they consider it a risk that they will take seriously. You can also investigate how they deal with some of the situations we've mentioned.

GotchasThere are some general things to check with a supplier. You could incur significant cost down the line if they're not clear. If the company is reputable, they should be happy to answer:

• How do they incorporate flexibility for internationalisation? (This is so that there is future flexibility for international growth with your product.) Some developers ignore the requirements for this; some go overboard on "what ifs". If you know you will never need this, it may be possible to save money by avoiding internationalisation support.
• How do they deal with maintenance issues after deployment (change, bugs, enhancements)? You need to know how they deal with after-contract work, and most importantly, how that will be priced compared to project work.
• What development methodology do they use? (When you find this out, look it up. Check that it is recognised, and fits with your own ethos and ways of working.)
• You might want to be able to code review at any point, at no additional cost, to ensure that code is both sufficiently commented and documented for future development – how would they make this possible? (You might not plan to do this as you'd need a specialist but it does ensure you can bring in independent vetting if needed.)
• How do they manage change control during a project? (If something changes at either end, or there's a technical surprise, how much cost will there be and how is it managed? This should be written into the project terms.)
• Is any of the work likely to be subcontracted? You want it to be contractual that work can only be subcontracted with explicit agreement. (I have seen companies take on work then subcontract to different, sometimes cheaper, teams or individuals. You need to guard against this.)

Contractual considerationsThat last point leads us onto contracts. With a software project there are a number of important things to consider in the contract itself, to make sure you don't get caught out. It's not that suppliers are often trying to fool you, but what is standard practice for them may not be obvious to a new customer. For example, you really need to make sure there are no hidden costs or licences involved that get exposed on delivery and that all Intellectual Property (IP) transfers to you at the end of the project. I’ve seen people get caught by suppliers who keep ownership of IP to lock the client in to them.
Let's look at some example clauses that might usefully be included. Of course, the notes below do not constitute legal advice and are not written in legal format: you should get all contracts drafted by a qualified lawyer. These are just some pointers to common things I've seen tripping people up:

• All intellectual property developed in the app, database and any other work performed under the contract becomes the sole property of the Client upon payment.
• Any connections with interfaces not owned by the Supplier will be agreed with the Client before use, along with price limits.
• Any connections with interfaces or services owned by the Supplier will be agreed with the Client before use. As part of that agreement, the Client will be granted a perpetual licence to use the interfaces at termination of the contract, at commercial rates agreed before the interface is used.
• Any hosting or third party services must be set up in the name of the Client and be available at project termination, including all data hosted, at no extra cost.
• At project termination, all code, data and documentation will be handed to the Client, in a state that can be compiled, and with sufficient documentation to understand it. This will be signed off by the Client as part of the final handover and payment.
• Any work subcontracted to another company or team will be signed off and agreed by the Client. Any work done under subcontract without agreement will be classed as a breach of contract.

Final thoughtsThis guide only scratches the surface of things you might consider. Different issues and considerations will appear for different kinds of project and supplier. The best approach is to engage a specialist who can help: on supplier assessment, business plan and contract analysis. Of course, when you're on a tight budget, that's not always possible - I've been there too. Hopefully I've provided some ideas on the types of things to assess when looking at potential partners.
The last thing to mention is that you never get this kind of analysis right 100% of the time. Even experts won't, so don't get too hung up on details - just make sure you're comfortable with the larger points. Good luck!