Independent research on bootcamp job placement claims

Question

Bootcamps have become an increasingly popular alternative for starting or transitioning to a career in tech over the past 5 years or so. As a result, a common question I get when interacting with beginners is whether bootcamps are indeed worth it.

Unfortunately, after conducting some preliminary research myself, I found it challenging to find hard data that supports a conclusion either way. Most online sources (such as coursereport.com) seem very marketing-y and don't seem to analyze the more existential question of whether bootcamps are a good idea in the first place, compared to other options. And of course, there are plenty of anecdotes, but anecdotes are not data.

Analyzing this situation is complicated by several other factors:

1. Bootcamps may be suitable for some, but not all: for example, professionals with an unrelated degree trying to transition to tech vs high school graduates trying to enter tech.
2. The quality of individual bootcamps can vary wildly (but then again, I suspect, though cannot confirm, the quality of university CS degrees can also vary just as wildly).
3. Bootcamps are incentivized to report skewed data. Even something as simple as "% of people who graduate with a job" can be complicated. (If the bootcamp hires you back as a TA, does that count? What if you get a job in a non-tech field? What if you get a job after a year of additional self-study?).

This brings me to my core question: What research has been conducted so far regarding how effective bootcamps are at helping their attendees obtain a full-time job in tech in a timely manner?

For this question, we can assume that the bootcamp attendees are people who either have completed a degree in an unrelated major and want to switch careers, or are people who are unable to attend a university for one reason or another.

An ideal answer would be links + summaries to reports that conduct a careful, measured, and data-driven analysis and seem to take into consideration factors like the ones I listed up above. I would prefer papers that have been accepted by a reputable, peer-reviewed academic journals or conferences if at all possible.

A nice bonus would be if the reports compared bootcamp success rates to other methods of switching careers (for example, getting a second degree or simply self-studying), but this isn't necessary.

For example, here is one paper that satisfies the above requirements. It'll apparently be presented at the upcoming ACM ICER conference later this month and draws some interesting conclusions -- the paper's main limitation is its relatively small datasize (it tracks only 26 students within a specific geographical location IIRC).

Answer 1

There's not much out there, unfortunately, and what there is is either suspect or sparse.

• CIRR has probably created the best data that we have, as they engage in 3rd-party verification. These results are actually rather positive for the industry, although participation is voluntary, and there aren't many Bootcamps that participate.
• The US Federal Government created the EQUIP initiative to convince schools to provide government-quality data in return for student attendees being eligible for financial aid. Only five organizations qualified. While I have been unable to locate the data itself, I would presume that those five organizations are probably fairly good.
• Course Report is an independent organization that creates an annual survey of outcomes, and it shows consistently good results for Bootcamps nationally, but all of the results are self-reported, so they are not entirely trustworthy. Also, the organization is very obviously friendly to the industry itself, with founders and employees who attended bootcamps themselves. Their "reports" simply look and read like ad copy.
• There is a recent initiative by the schools themselves to create more accountability, presumably in order to counteract rumors that the institutions are simply fraudulent diploma-mills.

If you are considering a Bootcamp, I would strongly recommend only considering schools that have good results with CIRR, as that will both discourage fraud, and encourage Bootcamps to partipate in the process.

Answer 2

I'm going to assume that you mean courses in which face to face presence is required, as opposed to MOOCs in which it is not.

There isn't likely to be much true research on such things, partly due to the recentness of their creation, but mostly due to the difficulty of formulating and answering appropriate questions. What data there is is pretty scattered and is subject to taint by financially interested parties.

However, opinion surveys are available, as well as reports by interested but not financially interested groups.

A set of reports can be found here: https://www.coursereport.com/resources/research-reports

Kyle Thayer and Andrew Ko recently interviewed a collection of students concerning barriers faced by bootcamp students: http://www.kylethayer.com/assets/papers/BarriersFacedByCodingBootcampStudents-Thayer-Ko.pdf. The study is small and so not definitive. It is also situated in time as these things change.

Of course, an online search for "value of programming bootcamps" will bring you a lot of opinions, but little research. The wisdom seems to be that bootcamps are good for making you better at your current job than they are for helping you move into a new profession. Employers may downplay the value of a bootcamp for valid or invalid reasons, but it still falls on the individual. If your resume shows no success in the new field, a bootcamp isn't likely to help you much.

But that is just another opinion, of course.

Still more opinion, but possibly useful: https://www.bloomberg.com/news/features/2016-12-06/want-a-job-in-silicon-valley-keep-away-from-coding-schools