Remember when beta testing was free?

If you buy something from a Verge link, Vox Media may earn a commission. See our ethics statement.

When I watched Amazon’s press event yesterday alongside my Verge colleagues, I was impressed with some of the new tech the company was introducing — but less than impressed with the way it introduced them. The deal is that, if you qualify for an “invitation,” you get the chance to buy — at a slight discount — a product that is still essentially in its beta phase. Oh, and please let Amazon know if anything goes wrong!

Before I continue, a brief history: Back in 2010, I applied for a Google pilot program for a new type of computer that eventually became the Chromebook. I was delighted when I actually made it into the program — according to Wikipedia, about 60,000 people did — and I received, free of charge, a somewhat weighty, black, and rather weird Cr-48 prototype laptop loaded with the new (and, at that point, only somewhat useful) Chrome OS. It was made very clear that this unit was for testing only (although it was, of course, written up in many tech publications). Those of us who received the Cr-48 were asked to use them and report on any problems we found or any general impressions of their usefulness. The first retail Chromebooks shipped about a year and a half later.

(Even after the test period was long over and much more streamlined Chromebooks were on the market, the Cr-48 machines continued to receive Chrome OS updates for several years. When they finally timed out, I rather sadly gave mine away to a friend who wanted to install Linux on it.)

Of course, most companies don’t give out tens of thousands of test devices for free the way Google gave out the Cr-48. If a company is producing software, it may offer beta versions — usually free — to its most active users, with appropriate warnings and perhaps a discount on the final version. Hardware manufacturers usually give out pre-production units to employees, developers, bloggers, and others so that any last-minute glitches can be discovered and fixed.

However, for the past few years, some vendors have been distributing what is essentially beta hardware — and asking us to pay for it. After all, why waste money giving out free hardware when your customers will pay you for the privilege?

Take the most recent example: Amazon’s new Astro “robot.” (I use the word “robot” with some ambivalence; as some of my colleagues have commented, Astro is basically a camera on wheels. Of course, the debate on what defines a robot is an old one.) Amazon seems to be wary enough of Astro’s current usefulness that it’s limiting the number of users and getting their feedback on how well it works. So it has announced that if you want this nifty new product, you need to apply for an invite. If you qualify, you can be one of the privileged few to pay $1,000 (as opposed to the $1,450 that it will cost the less adventurous masses) to see if the Astro really works.

And if it doesn’t?

Back in 2013, Google invited people to apply to try out one of its new beta projects, Google Glass. However, unlike with the Cr-48, you had to pay $1,500 for the privilege. For a little while, Google’s glasses were popular in the press, as activists and even Congress debated the privacy implications. But eventually, the glasses were pulled back from a retail item to something manufactured for construction, medicine, and other specialized fields. Somewhere, there are hundreds of original Google Glasses sitting in drawers, to be occasionally pulled out and reminisced over.

Sometimes I wonder, though: if Google had given out their Google Glass betas to a wide range of users instead of selling them to a small group of higher-income tech enthusiasts, would the outrage over the perceived privacy invasions have been quite so widespread? (Especially considering how much is recorded with our phones these days.)

Amazon seems to have developed this “pay to play” strategy into a fine art. Since September 2019, it has been allowing its customers to test its more innovative — or questionable — new equipment via a program it calls Day 1 Editions. Yesterday, along with the Astro, it added the Amazon Glow, a remote device for children that costs an “invitation-only” price of $250, and the Ring Always Home Cam, a security drone that flies around your house when there’s nobody else (including pets) there. The Always Home was actually introduced last year but was apparently not ready for testing at the time. Now you can try to get an invitation, and if you’re lucky, you can test one for $250.

If you read the intro on the Day 1 Editions page, the idea sounds very cool. “If you receive an invite,” it says, “you can purchase the product and get a chance to provide early feedback.” But you need to qualify for that invite — for example, if you want the Astro, you need to complete a survey that checks for glass mirrors or windows that extend to the floor, glass or translucent acrylic furniture, areas larger than 3,500 square feet, ramps, sunken areas, and solid black glossy flooring. That’s a possible way Amazon can potentially protect itself from negative feedback — the company can filter out any environment that it already has tagged as a possible weak point.

Amazon as Kickstarter
Of course, it could be said that what Amazon’s doing isn’t all that different from Kickstarter, where vendors ask people to help pay for the initial manufacturing run of their new product and, in return, get an early version at a lower cost.

But while Amazon has access to resources such as well-equipped and staffed development labs, funds to pay for testing, and enough employees so that new products can be distributed and tested in day-to-day circumstances, many (though, not all) of the enthusiastic vendors on Kickstarter do not. And while members of the Day 1 Editions club have nobody but themselves to blame if their new Astro falls down the stairs, a final Kickstarter product that is faulty will be trashed by its contributors.

In other words, Kickstarters ain’t Amazon.

Now, as a Prime member from way back and the owner of at least one Echo Show display, I like the fact that Amazon is developing new products. And with any luck, Astro robots could develop into something actually useful and popular. (After all, the original Amazon Echo was marketed as invite-only, and look how well it’s done.) Or they could end up in the backs of closets or on the workbenches of tech enthusiasts who enjoy hacking abandoned devices.

But here’s the thing: do you really want to shell out a grand in order to be a beta tester for a big company that has nearly endless resources? That decision is essentially up to you.