Histopathologic reassessment of melanoma and other melanocytic skin lesions excised in 2009 and 2018–2019
Main findings
In a histopathological reassessment of 196 melanocytic skin lesions from 2009 and 2018–2019, three pathologists frequently reported different diagnostic categories (kappa 0.45–0.50).
The proportion of reassessments that were in agreement with the original diagnostic category was 78.3 % for melanoma in situ and 81.2 % for superficial spreading or lentiginous melanoma.
The proportion of reassessments with a more serious diagnosis than the original was higher for histopathological slides from 2009 (29.8 %) than for slides from 2018–2019 (19.3 %).
The assessment of whether a pigmented skin lesion is benign or malignant is critical for the patient, further follow-up and epidemiological surveillance of cutaneous melanoma (1, 2). Final diagnosis of these lesions is based on histopathological findings, but histopathological diagnosis of pigmented skin lesions can be difficult and requires extensive experience (3).
Several studies have demonstrated low diagnostic precision and wide interobserver variation in histopathological examination of melanocytic skin lesions (4). In a large study by Elmore et al. with 187 pathologists from the United States, several lesions were even given a different diagnosis by the same pathologist in two examinations conducted a few months apart (4). In a study from a tertiary care centre in the United States, many skin lesions that were diagnosed as dysplastic nevi in 1988–1990 were considered to be melanomas 20 years later (5). This may suggest that diagnostic practice has evolved over time, and that the 'threshold' for diagnosing melanoma may have become lower for various reasons. It is conceivable that such a diagnostic drift has contributed to the sharp increase in the recorded incidence of cutaneous melanoma in recent decades (6, 7). This hypothesis has also been addressed in the Journal of the Norwegian Medical Association (8).
We are not aware of any similar studies conducted in Norway, even though Norway has a higher incidence and mortality for melanoma than most Western countries (9–11). Therefore, we wanted to investigate the agreement between different pathologists' assessments of histopathological slides of melanocytic skin lesions that had been removed and diagnosed several years previously, and to investigate the agreement with the original diagnoses.
Material and method
Using SNOMED codes, we identified all melanocytic skin lesions from patients over 18 years of age examined in 1999, 2009 and 2018–2019 in the Department of Pathology at Oslo University Hospital.
Using a computer-based algorithm, we retrieved randomly selected lesions in the following six diagnostic categories: 1) benign nevus, 2) irregular/dysplastic nevus, i.e. dysplastic nevus with moderate atypia, 3) nevus with severe atypia, i.e. dysplastic nevus with severe atypia, 4) melanoma in situ, 5) superficial spreading or lentiginous melanoma and 6) nodular melanoma.
These diagnostic categories were adapted from the study by Elmore et al. (4) based on diagnostic practice in Norway. Each category consists of several diagnostic entities in the World Health Organization's classification of skin tumours (12). The number of lesions in each category was chosen to give a varied and appropriate distribution of various melanocytic skin lesions, i.e. relatively few benign nevi without atypia and nodular melanomas, which do not usually pose diagnostic difficulties. The number in each category does not reflect prevalence. Slides from 1999 were excluded because most of the specimens from that year had been destroyed for reasons of space.
Histopathological slides from a total of 102 lesions from 2009 and 94 lesions from 2018–2019 were available and were assigned to three pathologists (AT, KG, KHL) from three different laboratories. The three pathologists are all specialists with three to ten years of experience in examining melanocytic skin lesions in routine diagnostic practice, but without formal qualifications in dermatopathology. Two of them mainly work exclusively in skin diagnostic evaluation; one assesses over 1,000 melanocytic skin lesions annually, the other two around 500. The slides were examined in random order in spring or autumn 2021. Each pathologist assessed the slides independently of the others and without knowing the original diagnosis, patient's age, location of the lesion or number of lesions in each diagnostic category. All assessments were then categorised separately for each pathologist according to the aforementioned diagnostic categories.
Agreement between the pathologists' assessments was estimated by proportion in the same diagnostic category with Wilson 95 % confidence interval (CI) (13) and Cohen's kappa (14) with 95 % CI (bias corrected, 5,000 replications) (15). Agreement between the pathologists' assessments and the original diagnoses was estimated by the proportion of assessments in agreement with the original diagnostic category with Wilson 95 % CI (13). The statistical analyses were performed in Stata, version 16.1 (StataCorp LLC, TX, USA).
Ethics
The study was assessed as a quality assurance project by the Regional Committee for Medical and Health Research Ethics in the Southern and Eastern Norway Regional Health Authority (project number 44624) and approved by the data protection officer at Oslo University Hospital (case number 20/04012).
Results
The agreement between the three pathologists' assessments is shown in Table 1. In the three pairwise comparisons, the pathologists selected the same diagnostic category for between 56.5 % and 60.7 % of the histopathologic slides, with Cohen's kappa ranging from 0.45 to 0.50.
Table 1
Agreement between three pathologists' diagnostic assessments of melanocytic skin lesions excised in 2009 and 2018–2019.
| Agreement | Cohen's kappa (95 % CI) | |||
---|---|---|---|---|---|
Comparison | n/n | % | (95 % CI) | ||
Pathologist 1 and pathologist 2 | 117/194 | 60.3 | (53.3 to 66.9) | 0.49 | (0.41 to 0.58) |
Pathologist 1 and pathologist 3 | 108/191 | 56.5 | (49.5 to 63.4) | 0.45 | (0.37 to 0.54) |
Pathologist 2 and pathologist 3 | 116/191 | 60.7 | (53.7 to 67.4) | 0.50 | (0.41 to 0.58) |
Table 2 shows the pathologists' assessments compared with the original diagnoses. Since each slide was assessed by three pathologists, a total of 582 assessments were performed (after removing 6 assessments that were not performed). The proportion of assessments in agreement with the original diagnostic category was 85.7 % (95 % CI 75.7 to 92.1) for benign nevus, 29.2 % (19.9 to 40.5) for dysplastic nevus with moderate atypia, 27.8 % (20.9 to 36.0) for dysplastic nevus with severe atypia, 78.3 % (70.4 to 84.5) for melanoma in situ, 81.2 % (73.7 to 86.9) for superficial spreading or lentiginous melanoma and 93.3 % (82.1 to 97.7) for nodular melanoma.
Table 2
Three pathologists' assessments of melanocytic skin lesions excised in 2009 and 2018–2019 compared with the original diagnosis. SSM/LM melanoma = superficial spreading or lentiginous melanoma.
|
| Assessments in 2021 (n) | Agreement | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
| Original diagnosis (n) | Total | Benign nevus1 | Moderate atypia2 | Severe atypia3 | Melanoma in situ | SSM/LM melanoma | Nodular melanoma | % | (95 % CI) | Not examined (n) | |
Total | Benign nevus1 | 24 | 70 | 60 | 3 | 2 | 4 | 1 | 0 | 85.7 | (75.7 to 92.1) | 2 |
| Moderate atypia2 | 24 | 72 | 9 | 21 | 23 | 16 | 3 | 0 | 29.2 | (19.9 to 40.5) | 0 |
| Severe atypia3 | 45 | 133 | 9 | 15 | 37 | 39 | 33 | 0 | 27.8 | (20.9 to 36.0) | 2 |
| Melanoma in situ | 43 | 129 | 1 | 2 | 5 | 101 | 20 | 0 | 78.3 | (70.4 to 84.5) | 0 |
| SSM/LM melanoma | 45 | 133 | 4 | 1 | 5 | 15 | 108 | 0 | 81.2 | (73.7 to 86.9) | 2 |
| Nodular melanoma | 15 | 45 | 1 | 0 | 0 | 0 | 2 | 42 | 93.3 | (82.1 to 97.7) | 0 |
| Total | 196 | 582 | 84 | 42 | 72 | 175 | 167 | 42 | 63.4 | (59.4 to 67.2) | 6 |
|
|
|
|
|
|
|
|
|
|
|
|
|
2009 | Benign nevus1 | 12 | 34 | 26 | 3 | 1 | 4 | 0 | 0 | 76.5 | (60.0 to 87.6) | 2 |
| Moderate atypia2 | 11 | 33 | 3 | 8 | 9 | 11 | 2 | 0 | 24.2 | (12.8 to 41.0) | 0 |
| Severe atypia3 | 26 | 77 | 4 | 6 | 18 | 22 | 27 | 0 | 23.4 | (15.3 to 34.0) | 1 |
| Melanoma in situ | 28 | 84 | 0 | 1 | 2 | 70 | 11 | 0 | 83.3 | (73.9 to 89.8) | 0 |
| SSM/LM melanoma | 20 | 59 | 0 | 0 | 3 | 10 | 46 | 0 | 77.9 | (65.9 to 86.6) | 1 |
| Nodular melanoma | 5 | 15 | 0 | 0 | 0 | 0 | 0 | 15 | 10 0.0 | (76.6 to 100.0) | 0 |
| Total | 102 | 302 | 33 | 18 | 33 | 117 | 86 | 15 | 60.6 | (55.0 to 65.9) | 4 |
|
|
|
|
|
|
|
|
|
|
|
|
|
2018–2019 | Benign nevus1 | 12 | 36 | 34 | 0 | 1 | 0 | 1 | 0 | 94.4 | (81.9 to 98.5) | 0 |
| Moderate atypia2 | 13 | 39 | 6 | 13 | 14 | 5 | 1 | 0 | 33.3 | (20.6 to 49.0) | 0 |
| Severe atypia3 | 19 | 56 | 5 | 9 | 19 | 17 | 6 | 0 | 33.9 | (22.9 to 47.0) | 1 |
| Melanoma in situ | 15 | 45 | 1 | 1 | 3 | 31 | 9 | 0 | 68.9 | (54.3 to 80.5) | 0 |
| SSM/LM melanoma | 25 | 74 | 4 | 1 | 2 | 5 | 62 | 0 | 83.8 | (73.8 to 90.5) | 1 |
| Nodular melanoma | 10 | 30 | 1 | 0 | 0 | 0 | 2 | 27 | 90.0 | (74.4 to 96.5) | 0 |
| Total | 94 | 280 | 51 | 24 | 39 | 58 | 81 | 27 | 66.4 | (60.7 to 71.7) | 2 |
1Without atypia
2Dysplastic nevus with moderate atypia
3Dysplastic nevus with severe atypia
The proportion of reassessments in which the diagnosis was more serious than the original diagnosis was higher for slides from 2009 than for slides from 2018–2019: 29.8 % (95 % CI 24.9 to 35.2; 90 out of 302 assessments) and 19.3 % (15.1 to 24.3; 54 out of 280 assessments), respectively (Table 2). The proportion of reassessments with a less serious diagnosis was also lower for slides from 2009 than for slides from 2018–2019: 9.6 % (6.8 to 13.5; 29 out of 302 assessments) and 14.3 % (10.7 to 18.9; 40 out of 280 assessments), respectively.
Discussion
This study revealed some considerable variation between different pathologists in the diagnosis made in histopathologic examination of melanocytic skin lesions excised in 2009 and 2018–2019, and frequent disagreement with the original diagnosis.
Interobserver agreement
Cohen's kappa estimates revealed unsatisfactory agreement between the assessments from the different pathologists. There was particular disagreement for lesions that were originally diagnosed as dysplastic nevus with moderate atypia and dysplastic nevus with severe atypia. This can be explained by differing interpretation of findings when diagnosing entities with overlapping morphological profile. Diagnostic evaluation of these lesions is largely based on a discretionary assessment of key morphological features. However, there was also disagreement in the assessments of lesions that were originally diagnosed as melanoma in situ and superficial spreading or lentiginous melanoma. The findings are consistent with the results of the study by Elmore et al. and other studies (4).
The variation in the histopathological assessments is probably not attributable to varying experience and expertise, at least not as the main factor. It is our opinion that the variation is instead a reflection of the fact that the diagnostic criteria for some melanocytic entities can be difficult to apply in standard routine diagnostic evaluation (4). These skin lesions are likely to be on a spectrum from clearly benign to clearly malignant with a biological and diagnostic grey area in the middle. This view is contrary to the common perception, particularly among lay people and patients, that tumours are either benign or malignant.
Agreement with the original diagnosis
The best agreement with the original diagnosis was for lesions that had been assessed as benign nevi and nodular melanomas. These two entities are each at the polar ends of the spectrum of melanocytic skin lesions. For the other diagnostic categories, agreement with the original diagnosis was to some extent considerably lower. These findings are consistent with findings in the aforementioned study from a US tertiary care centre (5). In our study, the histopathological slides were the same as those used for the original diagnoses, but we do not know whether several slides were examined at that time or whether the pathologist in question had consulted with other pathologists, which is the practice at some laboratories. Neither do we know whether additional methods were used at the time before making the final diagnosis.
Some lesions that were not originally diagnosed as melanomas were assessed to be melanomas, and vice versa. If these new assessments are the 'correct' ones, this means that some patients did not get a diagnosis of melanoma back then who should have, and that patients were told that they had melanoma when they actually did not. Assuming that the lesion was treated by radical excision with adequate margins, the patient would have been appropriately treated regardless of diagnosis, with a good prognosis in most cases. However, the emotional strain of receiving a diagnosis of melanoma on a dubious basis should not be underestimated.
One lesion that had originally been diagnosed as a nodular melanoma was assessed by a pathologist as being a Spitz nevus, a rare form of benign nevus. This type of lesion can be difficult to differentiate clinically and histopathologically from melanoma (16). One lesion that had originally been diagnosed as a benign nevus was assessed by a pathologist as being a superficial spreading melanoma.
Diagnostic drift?
It was our intention to investigate whether diagnostic practice has evolved over time, and we were informed that the department had access to histopathological slides, but only back to 1999. However, it turned out that most of the slides from that year had also been destroyed for reasons of space. Administratively and financially motivated decisions like these compromise the ability to study possible changes in diagnostic practice over time. In anticipation of upcoming digitalisation of histopathological slides, it is important to retain physical pathology specimens for research into potential diagnostic drift within cancer diagnosis.
The proportion of reassessments in which the diagnosis became more serious than the original diagnosis was higher for slides from 2009 than for slides from 2018–2019. This finding must be interpreted with caution, but does indicate that the 'threshold' for making a more serious diagnosis (including melanoma) may have become lower. The aforementioned study from a tertiary care centre in the United States suggests that diagnostic practice has evolved over time in that pathologists have become more aware of small morphological details that facilitate the diagnosis of a malignant lesion (5). This diagnostic drift may also be related to increasing awareness of misdiagnosis and compensation claims (5, 6, 17). These may also be contributing factors in Norway, although to a lesser extent than in the United States.
A possible change in diagnostic practice for skin lesions suspected to be melanoma is of particular importance for the interpretation of increasing incidence for cutaneous melanoma in Norway and other Western countries (7, 8). The increase in reported incidence of melanoma mainly applies to superficial spreading melanoma and has not been accompanied by a corresponding increase in melanoma-related mortality (18). This can be explained by the fact that early diagnostic evaluation and excision of thin melanomas has prevented further tumour growth, metastasis and death (19), but some diagnostic drift and overdiagnosis is still likely (7, 8). For some patients, a change in diagnostic practice may have an effect on how follow-up is conducted (1).
Implications
Disease classification systems, including those for cancer, are usually developed by leading experts based on prototypical cases and then adopted in routine diagnostic evaluation in all sections and at all levels of the healthcare services. Without validation of diagnostic criteria in this context, these classification systems, i.e. diagnoses, may turn out to be inappropriate with suboptimal validity and reliability (4). The World Health Organization published its most recent classification system of skin tumours in 2018 (12). In Norway, pathologists have largely continued to use poorly defined terms such as irregular nevus and nevus with severe atypia. These two terms correspond to what would be referred to internationally as dysplastic nevus with moderate atypia and dysplastic nevus with severe atypia, respectively. It is recommended that such lesions be excised in toto, mainly to ensure histopathological visualisation of the entire lesion (20).
Leading pathologists have recommended adopting a simpler classification system, Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis (MPATH-Dx), for the diagnostic evaluation of melanocytic skin lesions (21). This system uses only five diagnostic categories, roughly equivalent to the six used in our study. This type of simplification would enable improved communication between pathologists and clinicians, but does not solve the problem of possible overdiagnosis. The World Health Organization has introduced several new categories for melanocytic proliferations (12), such as melanocytoma, superficial atypical melanocytic proliferations of unknown significance (SAMPUS), melanocytic tumours of uncertain malignant potential (MELTUMP) and many others (22). There is much to suggest that these types of lesions should be considered as risk factors for melanoma rather than precursors to melanoma (23). However, it is difficult to study the clinical course of untreated dysplastic nevus and melanoma in situ because these diagnoses can only be confirmed by removal and histological examination of the lesion. Following radical excision of the lesion, the risk of further growth and metastasis is eliminated. The identification of cases in which excision of pigmented skin lesions is indicated can be improved by using dermatoscopy and confocal microscopy (24, 25). In the future, the use of artificial intelligence and machine learning may help simplify and improve the diagnostic evaluation of skin lesions suspected to be cancer (26).
Knowledge about the pathogenesis of melanoma and other melanocytic skin lesions is growing rapidly (27). New molecular biomarkers may potentially lead to more reliable and more reproducible diagnostic criteria. There will be an increased need for more expertise among pathologists in routine diagnostic evaluations. Our study and other studies indicate that difficult cases of melanocytic skin lesions should be examined by at least two pathologists (4). It is our opinion that consideration should be given to some degree of centralisation of dermatopathological activities in Norway, for example in the form of regional competence centres. Digitalisation of histopathological slides will give pathologists better opportunities to consult with colleagues at other laboratories. In several countries, dermatopathology is a separate subspecialty or a separate area of qualification for pathologists.
Openness about the fact that the diagnostic evaluation of pigmented skin lesions can be laden with uncertainty is just as important as increased expertise and knowledge, validated diagnostic criteria and new terminology (28). Medicine may have oversold the ability and opportunity of doctors to make definite diagnoses with a likely course and outcome for each patient. The task of conveying clinical judgement and uncertainty in the diagnostic evaluation of pigmented skin lesions can be difficult and requires professionalism and communication skills on the part of the doctor (2).
This study must be regarded as a pilot study. In our opinion, a larger study should be undertaken to investigate whether diagnostic drift may help explain the sharp increase in the incidence of cutaneous melanoma in Norway in recent decades (8).
Strengths and limitations of the study
As far as we know, our study is the first of its kind in Norway. The categorisation of diagnoses was adapted from a validated diagnosis system used in the study by Elmore et al. (4) and based on Norwegian practice. The selection of histological slides was performed to ensure an appropriate distribution of the entire spectrum of melanocytic skin lesions. The assessments were performed under similar conditions by three pathologists blinded to the original diagnosis and clinical details. The number of pathologists and lesions was lower than in some other studies, but the findings were largely consistent.
We did not investigate which histopathological assessments were 'correct', e.g. by having the slides examined by a panel of pathologists with specific expertise in melanocytic skin lesions. The growth phase (horizontal or vertical) of superficial spreading or lentiginous melanomas was not assessed. We have no information about whether the original diagnoses were made on the basis of several slides from the same lesion, or whether the pathologist in question had consulted with other pathologists. Neither do we know whether additional methods had been applied.
Conclusion
This study reveals some considerable diagnostic variation between different pathologists in their histopathological assessment of melanocytic skin lesions. This can be explained by differing interpretation of morphological findings, poorly reproducible diagnostic criteria and a lack of absolute distinction between benign and malignant lesions. Some evolution in diagnostic practice over time cannot be ruled out, with a more serious diagnosis being given for melanocytic skin lesions more frequently.
The article has been peer-reviewed.
Petter Gjersvik is senior associate editor of the Journal of the Norwegian Medical Association. He did not take part in editorial processing of the article.
- 1.
Nasjonalt handlingsprogram med retningslinjer for diagnostikk, behandling og oppfølging av maligne melanomer. 9. utg. Oslo: Helsedirektoratet, 2020.
- 2.
Gjersvik P. Melanoma is malignant, moles are benign. Tidsskr Nor Legeforen 2020; 140. doi: 10.4045/tidsskr.20.0486. [PubMed][CrossRef]
- 3.
Mihic-Probst D, Shea C, Duncan L et al. Update on thin melanoma: Outcome of an international workshop. Adv Anat Pathol 2016; 23: 24–9. [PubMed][CrossRef]
- 4.
Elmore JG, Barnhill RL, Elder DE et al. Pathologists' diagnosis of invasive melanoma and melanocytic proliferations: observer accuracy and reproducibility study. BMJ 2017; 357: j2813. [PubMed][CrossRef]
- 5.
Frangos JE, Duncan LM, Piris A et al. Increased diagnosis of thin superficial spreading melanomas: A 20-year study. J Am Acad Dermatol 2012; 67: 387–94. [PubMed][CrossRef]
- 6.
Titus LJ, Reisch LM, Tosteson ANA et al. Malpractice concerns, defensive medicine, and the histopathology diagnosis of melanocytic skin lesions. Am J Clin Pathol 2018; 150: 338–45. [PubMed][CrossRef]
- 7.
Welch HG, Mazer BL, Adamson AS. The rapid rise in cutaneous melanoma diagnoses. N Engl J Med 2021; 384: 72–9. [PubMed][CrossRef]
- 8.
Kristiansen IS, Bugge C, Førde OH. Bidrar overdiagnostikk til høye melanomtall? Tidsskr Nor Legeforen 2018; 138: 1007–10.
- 9.
Forsea AM, Del Marmol V, de Vries E et al. Melanoma incidence and mortality in Europe: new estimates, persistent disparities. Br J Dermatol 2012; 167: 1124–30. [PubMed][CrossRef]
- 10.
Cancer in Norway. 2020. Oslo: Kreftregisteret, 2021. https://www.kreftregisteret.no/globalassets/cancer-in-norway/2020/cin-2020.pdf Accessed 8.3.2022.
- 11.
2022. International Agency on the Research on Cancer. https://gco.iarc.fr/today/online-analysis-multi-bars?v=2020&mode=population&mode_population=countries&population=900&populations=900&key=asr&sex=0&cancer=16&type=0&statistic=5&prevalence=0&population_group=0&ages_group%5B%5D=0&ages_group%5B%5D=17&nb_items=10&group_cancer=0&include_nmsc=0&include_nmsc_other=1&type_multiple=%257B%2522inc%2522%253Atrue%252C%2522mort%2522%253Afalse%252C%2522prev%2522%253Afalse%257D&orientation=horizontal&type_sort=0&type_nb_items=%257B%2522top%2522%253Atrue%252C%2522bottom%2522%253Afalse%257D Accessed 8.3.2022.
- 12.
World Health Organization. Classification of skin tumors. 4. utg. Lyon: World Health Organization, 2018.
- 13.
Lydersen S, Fagerland MW, Laake P. Categorical data and contingency tables. I: Veierød MB, Lydersen S, Laake P, red. Medical statistics in clinical and epidemiological research. Oslo: Gyldendal Norsk Forlag, 2012.
- 14.
Lydersen S, Fagerland MW, Laake P. Diagnostic tests, ROC curves and measures of agreement. I: Veierød MB, Lydersen S, Laake P. Medical statistics in clinical and epidemiological research. Oslo: Gyldendal Norsk Forlag, 2012.
- 15.
Storvik G. Bootstrapping. I: Veierød MB, Lydersen S, Laake P. Medical statistics in clinical and epidemiological research. Oslo: Gyldendal Norsk Forlag, 2012.
- 16.
Brown A, Sawyer JD, Neumeister MW. Spitz nevus: review and update. Clin Plast Surg 2021; 48: 677–86. [PubMed][CrossRef]
- 17.
Carney PA, Frederick PD, Reisch LM et al. How concerns and experiences with medical malpractice affect dermatopathologists' perceptions of their diagnostic practices when interpreting cutaneous melanocytic lesions. J Am Acad Dermatol 2016; 74: 317–24, quiz 324.e1 - 8. [PubMed][CrossRef]
- 18.
Matthews NH, Li WQ, Qureshi AA et al. Epidemiology of melanoma. I: Ward WH, Farma JM, red. Cutaneous melanoma: etiology and therapy. Brisbane: Codon Publications, 2017.
- 19.
Chang C, Murzaku EC, Penn L et al. More skin, more sun, more tan, more melanoma. Am J Public Health 2014; 104: e92–9. [PubMed][CrossRef]
- 20.
Wiedemeyer K, Hartschuh W, Brenn T. Dysplastic nevi: Morphology and molecular and the controversies in-between. Surg Pathol Clin 2021; 14: 341–57. [PubMed][CrossRef]
- 21.
Lott JP, Elmore JG, Zhao GA et al. Evaluation of the Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis (MPATH-Dx) classification scheme for diagnosis of cutaneous melanocytic neoplasms: Results from the International Melanoma Pathology Study Group. J Am Acad Dermatol 2016; 75: 356–63. [PubMed][CrossRef]
- 22.
Ensslin CJ, Hibler BP, Lee EH et al. Atypical melanocytic proliferations: a review of the literature. Dermatol Surg 2018; 44: 159–74. [PubMed][CrossRef]
- 23.
Semsarian CR, Ma T, Nickel B et al. Do we need to rethink the diagnoses melanoma in situ and severely dysplastic naevus? Br J Dermatol 2022; 186: 1030–2. [PubMed][CrossRef]
- 24.
Carrera C, Marchetti MA, Dusza SW et al. Validity and reliability of dermoscopic criteria used to differentiate nevi from melanoma: a web-based International Dermoscopy Society study. JAMA Dermatol 2016; 152: 798–806. [PubMed][CrossRef]
- 25.
Pellacani G, Farnetani F, Ciardo S et al. Effect of reflectance confocal microscopy for suspect lesions on diagnostic accuracy in melanoma: a randomized clinical trial. JAMA Dermatol 2022; 158: 754–61. [PubMed][CrossRef]
- 26.
Esteva A, Kuprel B, Novoa RA et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017; 542: 115–8. [PubMed][CrossRef]
- 27.
Shain AH, Bastian BC. From melanocytes to melanomas. Nat Rev Cancer 2016; 16: 345–58. [PubMed][CrossRef]
- 28.
Elmore J. Joann Elmore: When diagnostic uncertainty hits home. Thebmjopinion 28.6.2017. https://blogs.bmj.com/bmj/2017/06/28/joann-elmore-when-diagnostic-uncertainty-hits-home/ Accessed 8.3.2022.