Defining the Optimal Threshold Scores for the Social Fobia Spectrum – Short Version (SHY-SV)

Background: The Social Phobia Spectrum – Short Version (SHY – SV) is a self-report instrument recently developed for the evaluation of a broad spectrum of social phobic manifestations, including both full-threshold and sub-threshold symptoms as well as atypical symptoms and personality traits. While the SHY – SV has proven a useful tool for the quantification of social phobic symptoms, it currently lacks a validated diagnostic threshold. 
Aim: This study aimed to determine the optimal cut-off score on the SHY - SV for identifying the presence of a clinical diagnosis of Social Anxiety Disorder (SAD).
Methods: the study included 104 subjects, of which 52 diagnosed with SAD and 52 healthy controls (HC). Trained psychiatrists conducted clinical diagnoses based on Diagnostic and Statistical Manual of Mental Disorders (DSM-5) criteria and subsequently assessed participants using the SHY – SV. A Receiver-Operating Characteristics (ROC) curve was used to find the optimal SHY - SV threshold score for the identification of clinical SAD.
Results: a cut-off score of 37 on the SHY – SV proved most effective in distinguishing individuals with SAD, exhibiting satisfactory levels of both specificity (0.981) and sensitivity (1.00).
Conclusion: our findings suggest that a SHY – SV score of 37 serves as the most discriminative threshold for identifying individuals with SAD.

INTRODUCTION

Social anxiety disorder (SAD) was first described at the beginning of the 20th century by Pierre Janet, who named it initially named it Social Phobia ¹. With the advent of the Diagnostic and Statistical Manual of Mental Disorders (DSM), SAD was initially grouped among specific phobias ² and then acquired its diagnostic dignity only in the third edition of the manual ³. Only in the DSM-IV the current denomination appeared, allowing a better understanding of the impairment and pervasiveness of the condition, clearly differentiating it from that of specific phobias ^4,5. Laslty, the most recent editions of the manual, the DSM-5 and the DSM-5-TR, have made some changes in the chapter dedicated to SAD, better defining the central characteristics of the disorder ^6,7. In particular, the new editions of the DSM have removed the distinction between the generalized and the specific form, the requirement to be over 18 years of age and the ability to recognize the excess and unreasonableness of the distress. Furthermore, a specifier for the “performance only” subtype was added to be used when SAD symptoms occur only when subjects have to perform in public ^6,7. To these days, SAD is defined as a marked and persistent fear or anxiety about one or more social situations in which one is exposed to judgment by others, lasting for more than six months ⁷. The same social situation is required to cause fear or anxiety almost every time it is presented, leading patients to actively avoid such situation. The fear or the anxiety perceived is disproportionate to the real threat, and the anxious symptomatology or the avoidant behaviors arising from it, are such as to cause significant discomfort or to compromise social or occupational functioning ⁷. The estimated prevalence of the disorder in the general population ranges from 1.9% to 13.7% ^8,9, although such data may even underestimate its extent, since it often remains undiagnosed due to its very nature, which increases the tendency to avoid contact with other subjects, including doctors, but also due to socio-cultural factors and prejudices on the acceptability of shyness, especially in females ^10,11. Typically, the symptoms of SAD begin in childhood ¹², persist throughout the entire school career and often lead to negative effects on both academic and work career ^13-16. Moreover, SAD often coexists with other mental disorders, in particular with mood disorders ^17,18 other anxiety disorders ^19,20, obsessive compulsive disorder ²¹, body dysmorphic disorder ²², alcohol and substance abuse disorder ^23-25. Since its conceptualization, a major concern in the field of SAD has been the definition of a diagnostic threshold ^10,11,26,27. Many researchers have suggested that SAD might be more properly classified as a spectrum of severity rather than as a distinct disorder based on a subjectively determined threshold ^11,27,28 and that the boundaries of SAD should be determined by its severity rather than by qualitative characteristics ²⁹. Following this conceptualization, several studies have hypothesized that SAD would be best classified as a dimensional continuum ^11,27,28. According to this literature, large subthreshold manifestations can coexist with major ones and mental disorder can be identified more easily using a spectrum model of psychopathology ^11,27,29. In this context, a self-assessment questionnaire named Social Phobia Spectrum-Short Version questionnaire was recently developed and validated in order to assess the wide spectrum of social phobic symptoms through the use of 139 items divided into five domains and one appendix. During its validation study, which lasted about 1 year, the questionnaire demonstrated an excellent internal consistency and test-retest reliability, along with strong convergent validity when compared to other dimensional measures of SAD and to the longer version: Social Phobia Spectrum Questionnaire ³⁰.

The aim of the present study is to determine the optimal cut-off score on the SHY-SV for identifying the presence of clinically significant SAD.

MATERIALS AND METHODS

Data have been collected between September 2022 and March 2023 at the Psychiatric clinic of the University of Pisa.

Study sample and procedures

The study sample was made of a total of 104 subjects: 52 subjects were recruited from out-patients afferent at the Psychiatric Clinic of the Azienda Ospedaliera Universitaria Pisana who had a clinical diagnosis of SAD, confirmed with the Structured Clinical Interview for DSM-5, Research Version (SCID-5-RV), and 52 were sex and gender-matched healthy controls recruited among health care and paramedical personnel. All subjects were required to have more than 18 and less than 70 years of age and to be available to sign an informed consent. The lack of mental disorders, with deep attention to the presence of SAD in healthy control (HC) was confirmed using the SCID-5-RV. From the sample were excluded individuals with language or intellectual impairments that could interfere with the evaluation, with relevant mental disabilities, not collaborating, or with persistent psychotic symptoms.

The study was conducted in accordance with the Declaration of Helsinki. All participants provided written informed consent, and protocol and assessment procedures were approved by the Ethics Committee of the Azienda Ospedaliero-Universitaria of Pisa the 19th of January 2023, with the approval code “23326_Dell’osso”.

Measures

Short Version (SHY-SV)

The SHY-SV is a self-report questionnaire tailored to assess the broad range of social phobic traits, ranging from the most common to the atypical and subtle ones, including also personality traits. The questionnaire consists in 139 dichotomous items organized in five domains and one appendix (Childhood and adolescence), and the score is obtained by counting the number of positive answers. The domains included in the questionnaire reflect various manifestations of the disorder and many situations in which the symptoms might exacerbate:

• interpersonal sensitivity: this domain investigates manifestations of hypersensitivity to criticism, to rejection and judgment, discomfort in being at the center of attention, the presence of reduced self-esteem, feelings of inferiority and difficulties in interpersonal relationships;
• behavioral inhibition: this domain focuses on social anxiety related behaviors such as low social participation, avoidance behaviors, submission’s attitudes and somatic symptoms;
• performance: this domain examines cognitive, somatic and behavioral manifestations related to ongoing social anxiety of daily activities such as phone calls, eating in public, work and other;
• social situations: this domain analyzes social anxiety signs and symptoms during interaction with other people;
• substance abuse: this domain investigates the possible use of tobacco, alcohol, benzodiazepines and other drugs, cannabis and other substances of abuse.

During its validation study, the SHY-SV demonstrated great test-retest reliability and convergent validity with other dimensional measures of SAD as well as a strong internal consistency 30. Since its validation, the questionnaire has been used to evaluate social phobic symptoms both in clinical and non-clinical populations ^31,32.

Statistical analyses

Student t-tests and Chi-square tests were use respectively to compare mean age and gender between the two diagnostic groups.

The Receiver-Operating Characteristics (ROC) curve was used to calculate the threshold value of the SHY-SV total score that could best discriminate subjects meeting the DSM-5 criteria for SAD from those without. According to the definition of accuracy, the ROC analysis was built on the basis of sensitivity and specificity for different cut-offs.

RESULTS

Our sample was made of 104 subjects (mean age: 39.12 ± 12.77), belonging to two diagnostic groups: the SAD group (N = 52; F = 25; M = 27; mean age: 40.12 ± 12.65) and the HC group (N = 52; F = 27; M = 25; mean age: 38.12 ± 12.93). As reported in Table I, the two diagnostic groups did not significantly differ from each other in terms of age (t = -0.797; p = .427) and gender (X² = 0.038, p = .695).

Results from the ROC analysis highlighted a score of 37 as the most discriminant cut-off of the SHY-SV total score for identifying subjects with SAD. Such value reported an AUC value of 0.989 (p < . 001), a sensitivity of 1, a specificity of 0.981 and a Youden Index of 0.981 (Tab. II and Fig. 1).

DISCUSSION

The aim of the present study was to determine a threshold score on the SHY-SV questionnaire for the discrimination of subjects with a clinical diagnosis of SAD. According to our research, the most useful threshold is a score of 37 on the SHY-SV, which denotes positive answers to 37 questionnaire items. Good levels of specificity (0.981) and sensitivity (1.000) were shown by this score. The SHY-SV could be a helpful tool in the daily clinical practice, thanks to its speed of administration, the low cost required and the short completion time. Furthermore, this questionnaire, by investigating both the striking features and the atypical and sometimes masked manifestations, can be of great help in screening the pathology which, historically, has often been overlooked. Indeed, although it is a relatively common disorder in the general population, affecting approximately 9% of children and adolescents and 12% of adults ^8,9, due to the very nature of the disorder, which increases the tendency to avoid contact with other subjects, including clinicians, and the presence of socio-cultural factors and prejudices on the acceptability of shyness, it is an often misunderstood and overlooked disorder, leading to an average latency of seventeen years between the beginning of the symptomatology to the seeking for help and treatment ^19,21. Furthermore, this diagnostic difficulty manifests itself to a greater extent in women, in whom the symptoms of anxiety present in social situations are often compatible with the classic subordinate role of mother and housewife wife which, although it is disappearing in today’s society, is always very rooted in particular geographical areas. In this context, the use of the questionnaire could therefore prove to be of great help to the clinician in evaluating social-phobic symptoms in the clinical population, with particular attention to women in whom it allows an objective evaluation of such symptomatology free of cultural and gender-based bias.

Furthermore, another difficulty in diagnosing social anxiety is represented by the fact that it often occurs in comorbidity with other mental disorders, in particular with mood disorders such as major depression, dysthymia or bipolar disorder, as well as with other anxiety disorders such as obsessive-compulsive disorder, generalized anxiety disorder, panic disorder, body dysmorphic disorder ^17-22. Also in this case, the use of the questionnaire can help in the evaluation of social-phobic symptoms that are present together with other psychopathological aspects, allowing for prompt recognition and guiding towards adequate treatment.

Although promising, the results of the present study should be seen in light of some limitations. For instance, the study sample was relatively small and with limited demographic characteristics. Secondly, although a threshold for differentiating between the presence and absence of social anxiety symptoms was discovered, differentiating between important sub-threshold features was not addressed. Lastly, due to the self-report nature of the questionnaire, it is possible that patients may over- or underestimate their perceived symptoms.

All being said, these results suggest that the SHY-SV is a valuable screening tool for SAD patients, helping to confirm and support a clinical diagnosis.

CONCLUSIONS

The best threshold value for the identification of SAD trough the SHY-SV questionnaire is a score of 37. which is associated with great levels of both specificity (0.98) and sensibility (1.00).

Conflict of interest statement

The authors declare no conflict of interest.

Funding

This research received no external funding.

Institutional review board statement

The study was conducted in accordance with the Declaration of Helsinki. All participants provided written informed consent, and protocol and assessment procedures were approved by the Ethics Committee of the Azienda Ospedaliero-Universitaria of Pisa the 19th of January 2023, with the approval code “23326_Dell’osso”.

Informed consent statement

Written informed consent was obtained from all subjects involved in the study.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

Conceptualization: LDO and BC. Methodology: BN, FG, and GM Investigation: BN, CB and FG. Formal analysis: G.M. Writing - original draft: BN and FG. Writing - review & editing: BC and LDO.

Figures and tables

FIGURE 1. ROC curve of the SHY-SV score for determining the presence of Social Anxiety Disorder. AUC = 0.989; Youden Index = 0.981; threshold score = 37.

TABLE I. Age and gender comparison between diagnostic groups.

		SAD (N = 52) Mean ± SD	HC (N = 52) Mean ± SD	t	p
Age		40.12 ± 12.65	38.12 ± 12.93	-0.797	.427
		N(%)	N(%)	Chi-Square	p
Sex	F	27(51.9)	25(48.1)	0.038	.695
	M	25(48.1)	27(51.9)

TABLE II. Area under the curve

Area	Standard error	P	C.I 95 %
			Lower Bound	Upper Bound
.989	.011	.000	.968	1.000