Developing effective strategies to prevent or treat coronavirus disease 2019 (COVID-19) requires understanding the natural immune response to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). We used an unbiased, genome-wide screening technology to determine the precise peptide sequences in SARS-CoV-2 that are recognized by the memory CD8⁺ T cells of COVID-19 patients. In total, we identified 3–8 epitopes for each of the 6 most prevalent human leukocyte antigen (HLA) types. These epitopes were broadly shared across patients and located in regions of the virus that are not subject to mutational variation. Notably, only 3 of the 29 shared epitopes were located in the spike protein, whereas most epitopes were located in ORF1ab or the nucleocapsid protein. We also found that CD8⁺ T cells generally do not cross-react with epitopes in the four seasonal coronaviruses that cause the common cold. Overall, these findings can inform development of next-generation vaccines that better recapitulate natural CD8⁺ T cell immunity to SARS-CoV-2.