Corey Adler @CoreyAdler He/Him/His Father Orthodox Jew Card-Carrying Libertarian .NET Developer Board Gamer Sci-Fi Watcher Soon to be Single You pick the order that works for you. Feb. 05, 2020 4 min read

OK, so it's taken me a couple of days to fully process the total meltdown that was the Iowa Caucus debacle.

As a professional software developer (of ~10 years) who has worked at multiple custom app development shops, allow me to break all of this down.

The articles that I've seen (WSJ cited below) indicate that Shadow Inc. (the shop hired to develop the app) were paid $63k by the Iowa Democratic Party and $58k by the Nevada Democratic Party.

These numbers represent an underbid of epic proportions.
 https://www.wsj.com/articles/the-shoestring-app-developer-behind-the-iowa-caucus-debacle-11580904037 

At the very minimum in these cases you're going to need 1 Project Manager (who gathers requirements, interfaces with the client, etc.), 1 Senior-Level Developer (who codes), and 1 Quality Assurance Engineer (who tests).

I spoke to one of my old bosses who confirmed the following numbers that would be billed for those people per month:

PM: $20k-$25k
Dev: $20k-$25k
QA: $13k-$17k

Which means you're looking at a bill of ~$60k per month for the very minimum that you're going to need.

For a supposedly 2 month project, you'd need ~120k.

"Hey, that's not that far off with what they paid! How can you call that an underbid?"

Because it completely ignores the realities on the ground.

Remember: I said that that was what was needed AT MINIMUM.

So here we go:

1) For an app that's going to be used by people you would assume (to be safe) are not tech savvy, you'll need a qualified UI/UX person to design it to be user friendly. That's going to add to the cost. Shadow does have a UI/UX person, so this was clearly done.

2) Each of the places that I've worked at have had, when they were smaller, a hesitancy to hire Junior developers. The reasons are obvious: Junior developers need more attention paid to their code, which takes up your Senior's time which clients then don't like being billed for.

Shadow is currently listed as having 2 Junior-level front-end devs (those who would work on the user interface) as well as a back-end intern (who would collaborate with the Senior dev on data related tasks).

These require more time from your Senior dev, and as such, more money.

3) I cannot emphasize this one enough:

Every app needs to be thoroughly and rigorously tested. Doesn't matter if it's an internal app that the outside will never see or an app that will be downloaded by millions. And in multiple different ways. For example, you'll need:

User Acceptance Testing: I get a few people who are expected to be my end users and get them to try and break the app every which way they can.

Load Testing: I simulate a high volume of users using the app at the same time to see if everything still works during high traffic.

Pen Testing: Simulating various attacks on the system, including attempts to corrupt the stored data, to certify that the app is secure.

Each of these are REQUIRED. Failure to properly Load Test, for instance, is the main reason why the ObamaCare website failed when it launched.

And for a public-facing, high profile app such as this one, that was going to be used for a whole state's Democratic Caucus (or 2 states, as Nevada was planning on using it as well until this happened), you're going to want even more time to do all of this testing.

More money.

So, as you can see, $111k is an incredible underbid of the actual cost for a company to make such a public-facing mobile app.

Which means that something needed to be cut in order to be done in the reportedly 2 months that it was being done in.

Care to guess what part got axed?

If you said #3, you are sadly correct.

I couldn't find anyone listed as working for Shadow as a QA Engineer. The only person I found with any QA experience at all was someone who was last a QA engineer for Apple...in 2011.

It came back to bite them.
 https://www.nytimes.com/2020/02/03/us/politics/iowa-caucus-app.html 

Even though the CEO claimed that they did test it, his own words beforehand betrayed him.

"The problem was caused by a bug in the code that transmits results data into the state party’s data warehouse."

Which would have been caught if UAT were done.

 https://thehill.com/homenews/campaign/481575-shadow-ceo-we-feel-really-terrible-about-iowa-caucus-app-flap 

Also, some users reported that night being unable to log into the system for the first time. Other users reported getting kicked out randomly from the app during the night.

Issues that would have been caught during Load Testing.

 https://www.latimes.com/business/technology/story/2020-02-04/clinton-campaign-vets-behind-2020-iowa-caucus-app-snafu 

So, considering that they clearly didn't do any of the first two testing methods that I listed, can we say, with any certainty whatsoever, that Shadow performed the 3rd one?

Probably not.

If so, what does that mean for the results still being published?

It means that we may never know if that data was accurate or compromised.

And considering how often Democrats end up nominating the winner of the Iowa Caucus, it's a question that may end up electing someone President.

Just think about that as they keep counting tonight.

UPDATE 2/6:

It appears that my initial suspicions were correct: They didn't do any kind of Pen Testing whatsoever. A vulnerability to this is the tech equivalent of givinga copy of your house keys to a stranger and then discovering that you were robbed.

 https://www.propublica.org/article/the-iowa-caucuses-app-had-another-problem-it-could-have-been-hacked 

UPDATE 2/7

Multiple issues have been discovered with the votes tallied during the Iowa Caucus, including some that are impossible to fix in some districts.

So I was wrong about the app potentially getting someone elected. It may have cost them instead.

 https://www.nbcnews.com/politics/2020-election/nbc-news-review-iowa-caucus-vote-finds-potential-errors-inconsistencies-n1132011 


You can follow @CoreyAdler.



Bookmark

____
Tip: mention @threader_app on a Twitter thread with the keyword “compile” to get a link to it.

Enjoy Threader? Sign up.

Threader is an independent project created by only two developers. The site gets 500,000+ visits a month and our iOS Twitter client was featured as an App of the Day by Apple. Running this space is expensive and time consuming. If you find Threader useful, please consider supporting us to make it a sustainable project.