Why do crashes cost you refunds and reviews?

Because Steam makes it easy to return a game that breaks early. Valve gives a refund for any reason if the player asks within 14 days of purchase and played less than two hours. Valve adds that players in some countries can have more rights to a refund when a game is faulty.

Valve also says that the refund notes in Steamworks can help you find bugs, balance problems or wrong expectations. A crash that hits many players in the first hour shows up in three places: the refund rate, the refund notes and the reviews.

Refund rate benchmarks are in our guide to Steam refund rates. This guide is about the crash side: how to count crashes, and how to connect them to the refund and review data that Steam gives you.

Sources: Steam Refunds policy; Steamworks Developer Refund Reporting documentation.

What are the Steam refund rules, exactly?

Two limits, and a few special cases. Valve refunds a game for any reason when the player asks within 14 days of purchase and has played for less than two hours. Outside those limits a player can still ask, and Valve says it will take a look. An approved refund is paid in full within a week.

The special cases matter when you plan a release. Playtime in Early Access counts against the two hours. For a pre-purchase, the 14 days start on the release date. Playtime in a Steam Playtest does not count, because a playtest is a separate app. So a playtest is the safe place to find crashes before buyers do.

Steam refund rules that matter for a build that crashes, from the Steam Refunds page (last updated 23 April 2024)
PurchaseRefund rule
GameWithin 14 days of purchase and less than 2 hours played
Pre-purchaseAny time before release, then 14 days from the release date and less than 2 hours played
Early Access or Advance AccessPlaytime before release counts against the 2 hours
DLCWithin 14 days, and less than 2 hours played in the base game since the DLC purchase
BundleLess than 2 hours of combined playtime for all items

Sources: Steam Refunds policy (last updated 23 April 2024); Steamworks Steam Playtest documentation.

What crash data should a game send?

Send two events: one for each error, and one for each run that ended badly. In Manalens, errors are normal events in the same table as your other events. So each error has the same player, session, build and platform as the rest of the play data.

The Godot SDK (4.5 and later) and the Unity SDK capture engine errors and exceptions. When a run crashes, the next launch sends app_unclean_exit in the session and build that crashed. A normal quit does not count, and a game that the operating system closed in the background does not count.

Manalens stores no log files, no native crash dumps and no symbols. An error event carries a message and a script stack trace of at most 1,024 characters each. For a segfault in the engine, keep Sentry or Backtrace for the dump, and use app_unclean_exit to count it.

Do not count on Steam for crash reports. Valve says that Steam Error Reporting is near its end of life, with limited support, and that it works only for 32-bit Windows applications.

The two crash events and what each one tells you
EventWhen it is sentUseful properties
errorAn error that your code reports, or a captured engine error or exceptionseverity, error_type, handled, fingerprint, breadcrumbs
app_unclean_exitAt the next launch, when the last run did not exit normally while the game had focusprevious_session_id, previous_build, previous_app_version

Sources: Manalens error tracking documentation; Steamworks Steam Error Reporting documentation.

How do I calculate crash-free rate?

Divide the crashed sessions by all sessions, and take the result from 1. Do the same for players. Crash-free sessions = 1 minus (crashed sessions / all sessions). Crash-free players = 1 minus (players with a crash / all players). On the Game health board, a crashed session has an app_unclean_exit event or a critical error with handled set to false.

A worked example with made-up numbers: build 1.0.3 has 12,000 sessions from 3,000 players in its first week. 240 sessions crashed, and they belong to 180 players. Crash-free sessions are 1 minus 240 / 12,000 = 98.0%. Crash-free players are 1 minus 180 / 3,000 = 94.0%. The player rate is lower, because one crash is enough to count a player.

Then count the refund risk. In the same example, 1,000 of the 3,000 players bought the game that week, and 70 of them crashed within two hours of their first event. So 7.0% of new buyers hit a crash while a refund was still open to them. In launch week, watch that number, not the 98.0%.

Do not compare rates from different tools, because they count in different units. Sentry counts sessions and users in a date range. Google Play counts daily users on Android, and marks an app for bad behavior when 1.09% or more of daily users see a crash. That is a mobile store rule. We found no crash threshold in the Steamworks documentation.

How three tools define a crash rate. The units differ, so the rates are not comparable.
ToolUnitDefinition
Manalens (Game health board)Sessions and players in the date range1 minus the share with app_unclean_exit or an unhandled critical error
Sentry (Release Health)Sessions and users in the date rangeShare of sessions not ended by a crash, and share of users with no crash
Google Play (Android vitals)Daily users, Android onlyShare of daily users with at least one user-perceived crash

Sources: Manalens error tracking documentation; Sentry Release Health documentation; Google Play Console Help (Android vitals); worked example with made-up numbers.

How do I find crashes in the first two hours?

Group new players by their first build, and count how many crashed soon after their first event. The query below uses wall-clock time from the first event, so it is a rough stand-in for playtime. Pick a date range that starts at your launch, so that the players in it are really new.

The Game health dashboard template gives the wider view: crash-free sessions, crash-free players, errors per session by build, and a Top errors table with the players each error affects. Rates over date ranges of different lengths are not comparable, so compare builds over the same number of days.

SQL (Explorer)

-- Players first seen in the date range, and how many crashed
-- within 2 hours (wall clock) of their first event, by first build.
SELECT
  build,
  count() AS new_players,
  countIf(crashed) AS crashed_in_2h,
  round(100 * countIf(crashed) / count(), 1) AS crashed_pct
FROM (
  SELECT
    anonymous_id,
    argMin(JSONExtractString(context, 'app', 'build'), original_timestamp) AS build,
    min(original_timestamp) AS first_seen,
    minIf(original_timestamp, event = 'app_unclean_exit') AS first_crash,
    first_crash > first_seen AND first_crash <= first_seen + INTERVAL 2 HOUR AS crashed
  FROM raw_events
  WHERE channel = 'app'
    AND anonymous_id != ''
    AND toDate(original_timestamp) BETWEEN {{date_from}} AND {{date_to}}
  GROUP BY anonymous_id
)
GROUP BY build
ORDER BY new_players DESC

Sources: Manalens error tracking documentation; Manalens Explorer documentation; Steam Refunds policy.

How do I find crash complaints in Steam reviews?

Pull your reviews from Valve's GetAppReviews method and filter them. The method needs no key, and it returns up to 100 reviews per page. Each review has the author's playtime at review in minutes, a refunded flag, a Steam Deck flag and, when the author shows it, the hardware: operating system, CPU, GPU and RAM.

Valve also lets you filter by playtime at review, by date range, by Steam Deck and by GPU or operating system. That is enough to ask a precise question, such as: which negative reviews came from players with under two hours who also refunded?

  1. Request negative reviews only, with playtime_max_hours set to 2 and a date range around your release.
  2. Search the review text for words such as crash, freeze, black screen and won't start.
  3. Count the matches that have refunded set to true.
  4. Group the matches by GPU and operating system, and compare with the platform split of your app_unclean_exit events.
  5. Read the build that was live on each review date, and check its crash-free sessions.

Sources: Steamworks IUserReviewsService documentation.

Which crashes should I fix first?

Fix the crashes that hit the most new players in their first session. The Top errors table shows one row for each error group with its count, players, first and last time seen and latest build. Sort by players, not by count, because one player in a crash loop can send many errors.

Then guard the next release. An alert counts one named event in each five-minute window and sends a Slack message or a signed webhook. For example, alert on at least 5 app_unclean_exit events with the filter app version is your new version. You get a message within minutes of a bad build, before the refund notes arrive.

  • First: crashes before the first save or in the tutorial.
  • Second: crashes that started with the newest build.
  • Third: crashes on one GPU or operating system that your reviews also name.
  • Last: handled errors that players do not see.

Sources: Manalens error tracking documentation; Manalens alerts documentation.

Where can I see crash-free sessions next to players?

Manalens is the analytics and growth engine built for games. Its Godot and Unity SDKs send errors and crashes as normal events, and the Game health dashboard template shows crash-free sessions, crash-free players and the top errors by build.

Sources: Manalens product documentation.

Frequently asked questions

Can I see which players refunded my game?

Not from Steamworks. Valve gives refund totals, a refund rate and grouped player notes, not a list of players. The Steam reviews API does mark each review with a refunded flag, so you can read the reviews of players who refunded. Line up your daily crash data with daily returned units instead.

Does a crash dump service replace crash analytics?

No, they answer different questions. A crash dump service such as Sentry or Backtrace shows where the code failed. Crash events in your analytics show which players, builds, campaigns and sessions the crash affected, and whether those players came back. Manalens counts native crashes with app_unclean_exit but does not store dumps.

What counts as a crash in Manalens?

An app_unclean_exit event, sent at the next launch when the last run did not exit normally while the game had focus, or a critical error with handled set to false. Crash-free sessions on the Game health board use both. A normal quit or a background close by the operating system does not count.

What is a good crash-free rate for a PC game?

We found no public Steam benchmark that we could verify. Google Play marks an Android app for bad behavior when 1.09% or more of its daily users see a crash, but that is a mobile rule. Compare each build with your last one over the same number of days, and watch new buyers in their first two hours.

How soon after a release can I spot a bad build?

Within minutes, if you set an alert. An alert counts one named event in each five-minute window. An alert on app_unclean_exit with an app version filter fires soon after players start the new build. Refund notes and reviews come later, often days after the damage.

Sources and implementation