Cost Estimation Transcript

We're talking about some general practices for developing cost estimates. I want you guys to get the idea that you're going to be making a lot of cost estimates throughout your career. It's not something that will necessarily be a small part of your job. It's going to be a large part of your job, and if you ever go into that "business for yourself" or if you ever start getting into management it's something you're going to do an awful lot of just because you need to go ahead and bring work in and that means you're going to be making cost estimates. As you're making cost estimates, I don't want you to think of those as just talking about what it takes to install something. We're also talking about the whole cash flow that's there. And so you're going to find yourself actually making quite a few estimates of cash flows like we've been doing with the income and cash flow statement. It's one of those ways that you can use to vet out ideas to see if they're actually going to work. And eventually you'll end up with, you know, a quick little spreadsheet that you fill out when you have an idea to see how things are penciling out. Now as far as these estimates go, there is not just one level of estimates. There are a couple of different levels and these have to do with the amount of effort that you put into them. They have to deal with the amount of uncertainty you have about the numbers that you put in them. And they have an awful lot to do with the amount of costs that is associated with them.

So let's start off with the three levels which are going to be the rough cost estimates, the semi-detailed cost estimates, and the detailed cost estimates. Rough estimates are for early stages. They're the kind of things that you can literally do on the back of an envelope or in a quick little spreadsheet. The idea is that these are supposed to be really cheap rough cuts. We are dealing with what you always approximately think it's going to cost. My favorite example of this one is (because I do quite a bit of survey research) is that I know a survey response is going to cost about $40. And I know that's what it's going to cost for a phone survey and you can just go ahead and multiply it from there. Now the real numbers will bounce around because it has to do with the length of the survey, it has to do with the number of responses, it has to do with the number of open-ended questions as opposed to closed-ended questions. But I know that number is just about right for a lot of things. The point with the rough cost estimates are is that there's going to be a load of uncertainty. Commonly between 30 and 60%. Now that'll be smaller if the kind of cost estimates that you make always are of a certain type. But they can't be that much smaller because there's always things that you don't know about the project. The other thing about these cost estimates is that they are classically going to be a little bit on the low side. The main thing is that you often don't forget something that saves you money. You tend to forget things which cost you money with some aspect you didn't think of: a part that you needed to get, a person that you needed to hire. You hadn't quite budgeted that, uh, put that in the budget. That's why we have contingencies.

Now the second level of cost estimates are the semi-detailed cost estimates. And these are things where you have put a little bit more thought into it. You've actually gone ahead and sticked [?] some time in. Kind of things where you may have some bids that are hanging around out there. Or a regular contractor that you use, and you've gone ahead and got them on board the kind of thing. The idea is that you do this in maybe some of your early detailed design phases. Uncertainty is typically a little bit smaller, but it is greater than that other level of cost estimation which are the detailed cost estimates. And these are when you're at the detailed design stage. This is where you have contracts, this is where you have blueprints, this is when you have quotes, typically it's very little uncertainty in that 3-5% range but, even then, you have contingencies because you know that there are expenses that you haven't considered. Now what I'm going to ask you to do, and what is probably excellent practice, is for you to take every single one of your cost estimates that you make and keep them. And keep them in a lab book. That's what you're looking at here. You're accumulating data on your ability to estimate costs. And you treat it as you would any other kind of scientific endeavor: you're looking for patterns within the data; empirical patterns that you can go ahead and use as leverage in order to make things better to actually improve your performance on making these cost estimates. While it may not look all that important to improve the cost estimates, imagine how much money you can save if you could get the same level of precision in your rough cost estimates as you currently get in your semi-detailed cost estimates. And semi-detailed cost estimates can be a lot more expensive. Imagine if you can go ahead and make it so that you have eliminated a bias that you have on every single bid that is equal to 10% of the project itself: that 10% that you are normally eating as a reduction in profits. That kind of thing is important to you, so you can see the benefits of going ahead and keeping a lab book and looking for details — patterns within the lab book.

Now, besides keeping your cost estimates, it's going to be very important to actually find out what the actuals were; what the ex-posts were. You can't just make a cost estimate and walk away; you're clipping out part of that feedback loop which is really important to you. So also within your lab book you need to add these actual costs, actual cash flows, so you can see how far off you are and start to figure out why you are off.

Well, now that you have your data accumulated after a while, we're talking about all your rough cost estimates, all your semi-detailed, all your detailed, and all the actual cost estimates, you need to start poring through the data to look for patterns. The first pattern that you should look for are looking for uniform differences between your rough estimates and your semi-detailed or your semi-detailed and your detailed and the actuals. And so you're looking at things that you're missing. Looking for things that you're consistently off on for whatever reason. Now, I have, because I have looked at my own data, seen where I make mistakes. I know I make mistakes in estimating the cost of taking data from other people since my work is primarily statistical and turning that data into something that I can use for analysis. And so this means having to do with figuring out the definitions of their columns because their codebook isn't as X, uh, up to the standards that I would like, or it has to do with irregularities in the data that they didn't document. Perhaps some of the folks were using Z time, some were on standard time, some were on Daylight Savings Time. I mean, all these little problems I had found out that I had underestimated how long it took me and time literally is money for me in this particular case. And so I know that whatever estimate I come up with, that I'm usually about, you know, I should multiply it by 2 or 3 in order to get what the actual number is supposed to be, and I know to do that now. I also know that I have underestimated the amount of time I need to devote to certain kinds of panic meetings that can happen at the end of a contract, before this report is supposed to go out to whatever board that we're dealing with, and so I know I have to budget that kind of stuff in because I usually think, "Oh, they won't panic again this time," and they always panic. You probably do things quite similar to this when you're dealing with household projects. You know that however amount of time that you expect to spend on it, it's always going to be 3 times as much. You always know that it takes 3 trips to the hardware store in order to finish a project. It's the same kind of idea — once you notice a pattern, you can go ahead and adapt to the pattern and make it so that your estimates are a little bit closer, even though you haven't changed something necessarily in your head; you're just following a rule that fixes it.

Now the next thing you should look at is just the pattern of cost estimates that you have within a project. And I say "the pattern," I'm looking at whether you go, like, a rough cost estimate followed by a rough cost estimate followed by a semi-detailed, detailed, or if you have something different. And the idea is that cost estimates are costly. They're expensive. Rough cost estimates? Yeah, you can do those on the back of an envelope. They take a little bit of time, you know, maybe it's a couple hours at the outside but there are some things that are relatively inexpensive and it's okay to have a lot of those. Semi-detailed cost estimates, I know in some domains, these are the kind of things that can cost you easily $5,000. I was dealing with a relatively small construction project and that's what I needed in order to figure out what I was going to do, what it was going to cost. $5,000. And detailed cost estimates: $40,000. These are the kind of things that you can spend a lot of time on. If any of you who are listening have ever responded to a Request for Proposals with a proposal, you can picture that there are 10 people working on this thing for 3 days. That's a nice chunk of money in order to produce a cost estimate. That's a lot of effort that goes into it. So you can see that the idea is to conserve the number of detailed cost estimates and semi-detailed cost estimates that you make and have a preference for the rough ones. Now, good patterns for estimates are things like one rough cost estimate, and that's a good pattern because you did the rough cost estimate, and you said, "No, we shouldn't do it," [snap] and you're done. You didn't go through [the] additional expense of giving semi-detailed or detailed. Another good pattern is perhaps something like a rough cost estimate and another rough cost estimate, which means you also didn't do it. What you should be striving for is that each cost estimate gives you the possibility of either rejecting [the project?] or moving on to the next level of detail. So it may be that you decide that you're going to do a kitchen remodel; you get a rough cost estimate for the kitchen that you try to remodel and you go, "Whoa, that's way too high." So you go back around, revise what you want, get another cost estimate, take a look at this and go, "Yes, this is something which is in the range I suspected it was going to be," and you go ahead and advance to the semi-detailed cost estimate, where any kind of weird stuff will tend to crop up, and then you can move on to the detailed cost estimates, if you like that. The point is [that] at each stage, what you're doing is hoping for either: move forward to the next stage, or reject and head out and not do the particular project. So again, the worst pattern you can see is something like rough, semi-detailed, detailed, followed by another rough, semi-detailed, and detailed. That's an extremely painful pattern because what it means is that you have had to revise the project in very expensive ways a couple of different times.

So, try to go ahead and have as few detailed cost estimates as possible. Now, the solutions to this are oftentimes client management issues. You'll see this a whole lot as something that's called "scope drift," but the key is to go ahead and make detailed notes on what it is that your client's actually looking for. You need to be the one that documents what's going on. You need to be the one that sends the summarizing memo. You need to be the one who goes ahead and writes up what kind of criteria is going to be used. And you need to do this because primarily it's something which is coming out of your pocket if things go bad on you. So please be the one to document it because you know it's going to be expensive. Don't be worried about a little scope drift; that's something which is common in the very early stages. That's why you have those rough cost estimates. But once you're at a detailed cost estimate, you know, changes beyond that are extremely expensive.

Now, moving on to looking at your ex-post and ex-ante estimates, which is the ultimate test of how good your estimates were — did they actually line up with reality? What you're able to see here is the hard numbers. If you think that the project was going to go ahead and cost you $100,000 and it came up costing $101,000, you probably did a really good job: that's impressive! And it shows you that you're doing things the right way and if you're consistently hitting numbers like that, that is really impressive. But if you notice that you're off by some large factors, [that] gives you an indication of skills you need to acquire. And that could be skills that you yourself need to learn or it could be that you need to bring other people in on the project. One of the revelations that I made to myself on the cost of some projects was, "You know what? We had some things that required PHP. I don't know PHP." I kind of thought I could hammer it out myself because I know enough programming languages that I could pick it up fairly easy. But, it meant that it took much, much, much more time than I thought it was going to in order to generate the kind of project that I needed. So I knew that in future projects, whenever I need to do this kind of cost estimation, I need to go ahead and bring in somebody who knows PHP, who can actually say, "Oh, this is something which is trivial," "Oh, this is something which is going to be more expensive." So you bring in those kinds of people.

The other thing which is useful when it comes to those ex-post estimates is: you don't necessarily have to deal with only your data. One of the things that's nice about bidding on projects is you often get to see what the winning bid looked like and you get to see what the winning costs were like. And so you can use all that information once it becomes public in order to improve your later cost estimates because commonly they see things that you don't or they have approaches (that you don't know of) to solving problems. So don't be afraid of using other people's data and then finally don't be afraid of using other sources. If you're lucky enough to be in an industry that has good costing manuals, use those when you can because those are actually usually extremely well-researched, and the estimates are extremely close.

Now the next thing you should look for is when you got beat up. And I don't mean this necessarily in a financial sense — it's when you got kind of, uh, T-boned by a criteria that you didn't expect to be important. And often this happens when you're dealing not with a single client that is a person but a client that's actually a group of people all together at the same time. And what you're really looking at here is the client being of two minds. It could be that they have conflicting criteria. One of them says it's gotta be red, the other one says it's gotta be blue. And you're getting dinged from one or the other because it's not red or it's not blue. Now, fixing this, again, is another client management problem. The key is to figure out who's fighting. And oftentimes it's fairly clear. The next step is to find out what they're fighting about. Hopefully, it is about the thing that they're discussing, but on occasion, it isn't. It could be that you have people that have simply been fighting because that's what they do and they've been doing it for 20 years, but you've got to get them focused on this criteria that you are being evaluated on, and if you can nail it down to those particular topics, meeting with them first individually to find out what the problem is, and then getting them together in a room to figure out what compromise they're going to come to on the criteria is very important. Key is: do not get yourself involved in their battle. Let them fight it out. You just need to go ahead and say, "I need a criteria out of you two because you two are in conflict. You need to go ahead and sort these things out so I can move forward." And again, the idea is to simply stay out of the way, and then make notes, document what that criteria is supposed to be that they'll eventually come to so you're not stuck being, you know, between the horns of a dilemma. Now, once you get in this process you will get to the point where there are some situations where you cannot win. These are expensive and they're also extremely common. My favorite example of these is a K12 example which is the DARE program, which is an anti-drug program that you'd see in elementary schools. Now, the DARE program is famous for making lots of people happy. The kids are happy because there are cops in the school, the cops are happy because there's additional pay that's involved there, the parents are happy because something's being done about drugs, and so on down the line, but research has shown, evaluation has shown that the DARE program doesn't do anything. There is no difference in the rate of drug and alcohol use depending upon whether you have the DARE program or not. It doesn't do anything at all, but it's expensive and it makes everyone happy. This is the kind of thing that can happen when you have your clients split up into many parts and you're busy trying to make them one. So please be aware that there is this possibility of a cost-increasing compromise.

And finally, start clocking in how long it took you in order to come to a decision. To a certain extent, this is like a test of character. I don't mean that in a good or a bad way; it just depends upon your temperament. Some people are very good at kind of glancing over the rough parts of the information and then just coming to a decision. That doesn't mean it's a right decision but they tend to make them quickly. Some people prefer to engage in an awful lot of analysis even to go ahead and come to some fairly trivial decisions. So if you're the kind of person that will do, you know, 3 days of research on a laptop so you can save $20 and get one additional feature, that's a little too much analysis: that wasn't good use of your time financially if you value it at all in a reasonable way. Some people have been known to just make completely gut decisions on things like a house or insist on almost no research when they decide on what they're going to do for college. And again, not necessarily good use of time. The key that I'm getting to you here is that the amount of effort that goes into the decision, should something have to do with the difference in value between your choices and your uncertainty about those choices. So let me go ahead and say that it's a bad idea to do $1,000 worth of research on a $10 question. Just don't do it — it doesn't make sense. And that has to do with the scale of the differences, but you also have to go ahead and be mindful of your uncertainty. You're going to find that it's often easier and cheaper to come to a decision if you decide on a criteria before it is that you start collecting data. My best example I have of this one is: I had some clients that were looking at, and they originally asked for, the fraction of households in the population that had the energy-efficient version of a certain kind of common household object. And they wanted to know what fraction of the households had these things and I was [like], "Well, what are you trying to do here?" It's, well, they wanted to decide whether they were going to discontinue a subsidy. And so they needed to know what fraction of households had these, and that's kind of a silly thing. Well, I said, "Well, wait a minute. That's kind of expensive to do. Let's go ahead and get you to a decision criteria first and we can maybe do this a little bit cheaper." And it turned out, if more than 20% of the population had these items in their household, they were perfectly willing to terminate the subsidy. And if you know anything about surveys, it's a lot easier to find out if the number is more than 20% than it is to find out that it is, you know, 50% plus or minus 2%. It's a lot cheaper. Oftentimes it can be 10 times as cheap depending upon what the numbers look like. And we ended up doing something like that and so it was a way of minimizing the decision-making cost. Now, my advice to you is: if the difference between the choices is something that's relatively small, flip a coin. It doesn't matter. Don't engage in any more effort than what you estimate to be the difference is. If your uncertainty is large, that's that kind of situation where you either need to come to a decision criteria before you make your decision, or what you need to do is spend the money to resolve your uncertainty. And that should be just about the last advice I have for you on making your cash flow and cost estimates.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License