What should I measure in a playtest?

Measure starts, session length, the point where testers quit, errors and crashes, and returns. Each one answers a different question, and each one comes from a small set of events. Surveys and interviews tell you why. Events tell you how many and where.

The playtest measures, the event behind each one and the question it answers.
MeasureEventQuestion it answers
Startsgame_open with is_first_openHow many testers launched the build?
Session lengthsession_end with duration_sHow long does a session last?
Progresslevel_start, level_complete, level_failWhere do testers stop or fail?
Errors and crasheserror, app_unclean_exitWhat share of sessions crash?
Returnssession_start on a later dayDo testers come back the next day?
IntentA click on your wishlist or feedback buttonDo testers want more?

Sources: Manalens app events, metrics and error tracking documentation.

How does Steam Playtest work?

Steam Playtest is a separate child app of your game. It has no store page of its own and shows as a section on the store page of the base game. Because it is a separate app, it does not affect the wishlists, reviews, refunds or achievements of your main game.

Players request access on your store page. With limited sign-up, the default, you grant access in batches, and Steam selects players at random from the pool. With open sign-up, Steam adds players as they ask. You can also target countries, for example to manage server latency.

Playtests must be free. An experimental option lets each player invite friends: you choose 1, 2 or 3 invites per player, and players must have been friends for 30 days. The Steamworks example: 5,000 players with three invites each can send up to 15,000 invites.

Steam emails a player when you add them to the playtest. It does not email anyone when you set a build to playable, so Valve recommends an announcement for that. A playtest is not confidential: Valve says testers are not under a nondisclosure agreement and should not expect secrecy.

The Steamworks Playtest documentation does not describe a metrics report. Its dashboard shows the users who have access. For feedback, Valve says that collecting it is up to you, and notes that a number of developers put a large feedback button on the main menu. Plan your own measurement before the first batch of testers.

Steam Playtest facts, from the Steamworks Steam Playtest documentation (read 19 November 2026).
TopicWhat Valve documents
PriceFree for players and developers. You cannot charge for access or sell playtest keys.
Store pageNone of its own. The sign-up shows on the store page of the base game.
Sign-upLimited (the default, you add players in batches) or open (players are added as they ask).
Who gets inSteam picks at random from the players who asked. You can limit by country.
Friend invitesExperimental. 1, 2 or 3 invites per player, only to friends of at least 30 days.
KeysStandard release keys only. Above 50,000 keys, Valve treats the test as an open beta.
Main gamePlaytime, wishlists, reviews and refunds are separate. A tester cannot review the main game.
EndSet the playtest to Not Playable. A reset removes all testers and cannot be undone.

Sources: Steamworks Steam Playtest documentation.

Should I use Steam Playtest, keys or a demo?

Use Steam Playtest for most closed tests, release override keys for a confidential test, and a demo when you want everyone to play. Valve calls Steam Playtest the preferred way to run a closed beta. It names two common ways to test with people outside the team: Steam Playtest, and release override keys on the App ID of the main game.

Release override keys are limited to 2,500 per app, and they keep access to the base game until you ban them. Valve points to them when you need a nondisclosure agreement, because you run the sign-up yourself. Playtest keys are easier: you can ask for larger batches, and they work even before the store page is live.

A demo is public and has no gate. Steam Next Fest needs one. A playtest does not change your Next Fest eligibility, but Valve does not recommend a live playtest during the fest, because it splits your players between two builds. If you want to charge for an unfinished game, Valve points to Early Access, which is for games in a playable alpha or beta state.

Three ways to put a build in front of players on Steam, from the Steamworks documentation (read 19 November 2026).
OptionWho can playGood forLimit
Steam PlaytestPlayers you add, or anyone with open sign-upClosed and open tests before releaseMust be free; not confidential
Release override keysPlayers you send a key toA confidential test with your own sign-up2,500 keys per app
DemoEveryoneSteam Next Fest and wishlistsPublic; no control over who plays

Sources: Steamworks Steam Playtest, Testing On Steam, Steam Next Fest and Early Access documentation.

How many testers do I need?

About 5 per round to find problems, and 20 or more to trust a number. Both figures come from usability research on websites by Jakob Nielsen, not from games, so read them as a starting point.

Nielsen found that the first 5 test users find about 85% of the usability problems, because a single user finds 31% on average. He advises three small rounds of 5 with a fix after each one, not one round of 15. For a game, that means: watch 5 people play the first 30 minutes, fix what stops them, then invite the next 5.

Numbers need more people. Nielsen writes that 20 users give a reasonably tight confidence interval for a usability metric. With 20 testers, one tester moves a rate by 5 percentage points, by our arithmetic. So a funnel from a 20 person test can show a wall, but it cannot show a small change between two builds.

This is why limited sign-up fits a playtest well: add a small batch, fix, then add the next batch on a new build.

Sources: Nielsen Norman Group (18 March 2000; 25 June 2006); Steamworks Steam Playtest documentation.

How do I set up analytics for a playtest build?

Install the SDK, give the playtest build its own build value, turn the SDK off in the editor, and add a few progress events. The Godot and Unity SDKs send game_open, session_start, session_end, errors and crashes for you.

  1. Install the Godot addon or the Unity package, and copy the Publishable key from your project settings. Never ship a Secret key in a build.
  2. Call init with build set to the playtest build, for example 571, and storefront set to steam.
  3. Set enabled to false in the editor, so your own test runs do not mix into the results.
  4. Track level_start, level_complete and level_fail with level and attempt, or one event per checkpoint.
  5. Run the build once and check that game_open and session_start appear in the activity feed.
  6. Give each new playtest build a new build value before you send it to testers.

GDScript

Manalens.init({
    "publishable_key": "YOUR_PUBLISHABLE_KEY",
    "endpoint": "https://YOUR_COLLECTOR_ENDPOINT",
    "app_version": "0.9.0",
    "build": "571",
    "storefront": "steam",
    "enabled": not OS.has_feature("editor"),
})
Manalens.track("level_start", {"level": 2, "attempt": 1})

Sources: Manalens Godot SDK and Unity SDK documentation.

How do I find where testers quit?

Build a funnel from the first launch to your later checkpoints and read the largest drop. A funnel in Manalens has two to five steps and counts people, not events. The default conversion window is 24 hours, which suits a first session.

The Levels table in the Game KPIs dashboard adds the fail rate per level: fails divided by completes plus fails. A level with a high fail rate and a large funnel drop is a strong lead. User paths then show what testers did before they left.

Conversion insights under a funnel list the events that go with completion. They show correlation, not cause, and a row needs at least 10 people. Small playtests often show no rows.

Sources: Manalens funnels, user paths and metrics documentation.

How do I compare two playtest builds?

Filter or break down every report by build. The App filter group has Build, so a filter such as Build is 571 limits a report to one build. Funnels, top events and retention take a breakdown by app_build.

The Game health dashboard template shows crash-free sessions, crash-free players, errored sessions and errors per session by build. A crash counts when the next launch sends app_unclean_exit in the session that crashed. Manalens does not store native crash dumps, so keep a crash reporter such as Sentry or Backtrace for stack traces.

Compare builds over date ranges of the same length. A longer range has more sessions for each player, so the rates change.

Sources: Manalens app events and error tracking documentation.

What does a playtest report look like?

One page: the build, the question, the funnel, stability, returns, the top survey answers and the decisions. The table below is a sample report for a made-up game. Every number in it is an example, not a benchmark.

Read the sample from the top. 148 of 200 invited players launched the build. The largest loss is at the first boss: 121 testers finished the tutorial, but only 77 beat the boss, and the level has a 64% fail rate. Crashes are a second problem, but a smaller one. So the decision is about the boss, and the next build tests that one change.

Write the decision in the report, with the build that will test it. A report without a decision is only a list of numbers.

  1. Write the question before the test starts, and one number that would answer it.
  2. After the test, filter every report by the playtest build.
  3. Fill the funnel lines first, then stability, then returns.
  4. Add the three most common survey answers in the words of the testers.
  5. End with at most three decisions, and name the build that will test each one.
Sample playtest report for a made-up game. All numbers are examples.
LineExample valueWhere it comes from
Build and datesBuild 571, 7 daysThe build value in the SDK
QuestionCan new players beat the first boss?Written before the test
Players given access200Steam Playtest dashboard
Launched the build148 (74% of 200)game_open with is_first_open
Finished the tutorial121 (82% of 148)Funnel step: your tutorial_complete event
Beat the first boss77 (52% of 148)Funnel step: level_complete, level 2
Fail rate at the first boss64%Levels table: fails / (completes + fails)
Crash-free sessions96.5%Game health dashboard, build 571
Came back the next day53 (36% of 148)Daily retention, D1
Survey answers61 (41% of 148)Your survey tool
Top survey answerThe boss gives no hint before its big attackOpen question in the survey
DecisionAdd a warning animation; test again in build 572The team

Sources: sample report with made-up numbers; our own method.

Will playtest numbers match my launch?

No. Testers chose to sign up, and friend invites bring in people who know each other. Use playtest numbers to compare one build with the next, not to forecast sales.

Retention in a short playtest is also limited. A daily retention chart in Manalens shows days 0 to 30, and periods that are not complete show a dash, not a low number. A one-week playtest can show D1, but not D30. Rolling retention counts a tester who returns on that day or later, so it suits a short test with few testers.

Session length needs the same care. The Game KPIs dashboard shows the average duration_s of session_end events, and a few very long sessions pull an average up. Read it next to the funnel, and compare one build with the next, not with a number from another tool.

Sources: Steamworks Steam Playtest documentation; Manalens retention and metrics documentation.

What should I not collect from testers?

Do not put personal data in event names, properties or error messages. The SDKs send a random anonymous ID, the session ID and the app context. They send no device ID, no hardware details and no IP, and the collector derives the country and does not store the IP.

If your playtest needs consent before tracking, start the SDK with consent_required. Then it records no events and writes no ID until set_consent(true).

Sources: Manalens Godot SDK and app events documentation.

How do I start measuring my next playtest?

Manalens is the analytics and growth engine built for games. Install the Godot or Unity SDK, tag the playtest build, and read the first session funnel, Game health and returns by build in one project.

Sources: Manalens product documentation.

Frequently asked questions

Does a Steam Playtest affect my main game reviews?

No. Steam Playtest is a separate child app of your game. According to the Steamworks documentation, it does not affect the wishlists, reviews, refunds or achievements of the main game. It has no store page of its own and shows as a section on the base game store page.

What is the first metric to check after a playtest?

Check how far testers got in the first session. Build a funnel from game_open to your first few checkpoints, read the largest drop, then check the fail rate of that level and the crash-free sessions of the build. Fix that point before you add content.

How do I keep my own test runs out of the playtest data?

Set enabled to false in the editor. In Godot, use enabled: not OS.has_feature("editor"). In Unity, use Enabled = !Application.isEditor. With enabled set to false, the SDK sends nothing, writes no files and keeps no ID, so only builds that testers run send events.

Can I measure D7 retention in a playtest?

Only if the playtest stays open for at least seven days after testers start. Manalens shows a dash for periods that are not complete. For a short test, read D1 and use rolling retention, which counts a tester who returns on that day or any later day.

Sources and implementation