911 Calls received from all five boroughs in NYC via FDNY dispatch from April 3, 2020, to October 2, 2020, were screened for specific inclusion and exclusion criteria by trained emergency medical dispatchers (EMD) using a computerized algorithm (Fig. 1). Patients aged 5 to 59 were eligible given the patient was the caller (or an adult parent calling for a minor present) and the patient was located in a safe location where a telemedicine encounter could be performed (i.e., patient is not on the street). Inclusion criteria for transfer were based on predetermined call types determined by the EMD. All FC (fever and cough; which could be added to any call type concerning for possible COVID-19 infection) were eligible. Respiratory complaints (RESPIR) were eligible if designated FC and did not meet any of the following exclusion criteria: can speak in full sentences, no associated chest pain, no cardiac, asthma, or chronic obstructive pulmonary disease history, and has never been intubated in the past). Minor medical complaints not fitting into other call type categories were eligible, typically designated by SICK (including, but not limited to sore/abscess/boil, chills, dizziness/vertigo, fever, diarrhea, constipation, nausea, vomiting, headache, other non-chest pain, rash, sickle cell crisis, or swelling) or SICMIN (including, but not limited to earache, malaise, genital problem, medication refill, respiratory symptoms including cough, congestion, or cold symptoms, toothache, or urinary problems) call types.

Eligible calls were transferred by the EMD to a hospital-based hotline. Callers were asked “what hospital do you normally go to for your healthcare?” and transfers were based on the health system affiliated with that hospital; for patients without an established healthcare provider, callers were randomly assigned to a telemedicine service. Callers were then connected to a team of nine staff nurse practitioners (NP) or physician assistants (PA) via telephone for further remote triage (RT). This line was staffed when call volumes were highest: 12 hours per day, 5 days per week to allow for provision of medical advice or transfer to telemedicine provider as appropriate. Triage NP/PAs were able to dispatch ambulances using the FDNY incident number for calls deemed not appropriate for telemedicine after further RT. Triage NP/PAs could provide medical advice, prescribe medications, schedule follow-up appointments and/or facilitate initiation of a video visit with an emergency physician staffing our institutional virtual urgent care service line. Workflows were provided for NP/PAs to assist callers with downloading and scheduling virtual urgent care appointments. Follow-up calls were attempted on all patients within 48 hours of initial 911 call by a dedicated registered nurse with care management certification (RN) during the first 3 months of the program to assess patient outcomes, provide additional medical advice or clinical navigation, and collect information on subsequent healthcare utilization.

Data on age, caller location, medical comorbidities, history of present illness, and call outcomes were entered directly into the study database using a secure web-based survey form during the initial call by NP/PAs performing RT. Follow-up calls attempted by the RN used a structured telephone script and data collection form to evaluate callers the persistence of symptoms, the type and timing of additional medical care sought including repeat 911 activation, ambulatory clinic visits, ED visits, and hospital admissions including intensive care unit admissions. Data were reviewed by three clinicians not involved in the initial call-taking process (WH, ML, and BD) and an Emergency Severity Index (ESI) score was assigned by each provider based on chart review, agreement between assigned ESI levels was assessed, and discrepancies were averaged. As RT did not include evaluation of vital signs, ESI was assigned based on the presence of immediate threat to life or limb requiring emergency intervention (ESI level 1), high risk criteria such as altered mental status, severe distress or pain (ESI level 2), and the number of anticipated resources required such as laboratory test, radiology or intravenous medication (many, one, none indicated ESI level 3, 4 or 5 respectively). Since abnormal vital signs are typically used to up-triage ESI 3 to ESI 2, we assumed any patients reporting significant respiratory or cardiac symptoms would be triaged to ESI2. Study data were collected and managed using a secure interface termed: REDCap (Research Electronic Data Capture). REDCap is a secure, web-based software platform designed to support data capture for research studies, providing (1) an intuitive interface for validated data capture; (2) audit trails for tracking data manipulation and export procedures; (3) automated export procedures for seamless data downloads to common statistical packages; and (4) procedures for data integration and interoperability with external sources [9,10]

Caller demographics reported medical comorbidities, caller origination, chief complaint organ system, and disposition were computed as frequencies. Reasons for ambulance dispatch after remote are reported along with type and frequency of follow-up care after initial call. Weekly call volumes were compared to COVID case counts using data publicly available from the New York City Department of Health [2]. Chi-square and analysis of variance tests to examine differences in characteristics between patients in early and late 2020, coinciding with dropping COVID-19 case numbers during the summer and fall of 2020 [2]. Differences in the reason for ambulance dispatch between early and late 2020 emergency calls were examined by odds ratios (OR) obtained from multivariable logistic regression models. Crude differences are reported in addition to results adjusted for patient demographics (age, sex, and borough). Comparisons between these time periods for ESI were reported as mean differences from t-tests. The inter-rater reliability of three individual ESI ratings was evaluated using Light’s kappa [11,12]. An alpha of 0.05 was used for hypothesis testing and all statistical analyses were performed using R v4.1.112. This study was approved by the Weill Cornell Medical College Institutional Review Board (IRB) (No. 21-09023955). Informed consent was waived by the IRB.

Four hundred and fifty-seven calls were transferred to our telemedicine service from April 3, 2020, to October 2, 2020 (Fig. 2). Call volume varied by week with an average of 3.8 calls per day (median, 4; range, 0-14). The highest daily number of diverted calls (n=14) were transferred on 4 of 27; multiple dates had no calls diverted to telemedicine. Three calls (0.7%) had no interaction between provider and patient and were removed from the analysis. As shown in Table 1, calls originated throughout all boroughs across NYC, however the majority were from the Bronx (16.5%), Brooklyn (14.5%), and Manhattan (13.2%); fewer calls were received from Queens (9.7%) and a small proportion (1.5%) were from Staten Island.

The average age of the 454 included callers was 43.3 years (interquartile range, 31.0–54.1 years). Female callers made up slightly less than half (46.9%) and 3.1% were pediatric (less than 18 years old). Most patients (85.5%) had at least one medical problem: 22.9% had hypertension, 16.1% had diabetes, and 5.3% had heart disease, 7.7% were immunosuppressed and 11.7% and 4.8% had lung and kidney disease, respectively. Comparing demographics between the early phase of the initial COVID-19 pandemic in NYC and the later phase, callers during periods of lower COVID case counts were less likely to have significant comorbidities, specifically hypertension, diabetes or be immunosuppressed.

Table 2 compares presenting complaints, ESI triage level and call outcomes between the initial and late phases of the first COVID-19 wave in NYC. Overall, 18.5% of calls were determined to be related to COVID after RT with a statistically significant decrease in COVID-19-related calls (26.0% to 10.8%, P=0.004) between early months of the initial pandemic wave and the latter. Respiratory-related calls also dropped significantly between the two time periods (P=0.008). There were 8.6% more musculoskeletal-related complaints during the second period relative to the first while all other complaint categories remained static: 18.3% were related to the gastrointestinal system and 17.6% related to the musculoskeletal system. Neurologic (13%), respiratory (12.8%), and cardiovascular (10.4%) were the next most common complaints followed by ear/nose/throat, genitourinary, and dermatological, which were each less than 10%.

ESI was assigned retrospectively. One patient had an ESI of 1 and 4.4% were assigned ESI of 2. Most patients (56.8%) were assigned ESI 3, followed by ESI 4 (28.9%). Few patients (4.8%) were assigned an ESI level of 5. Agreement between reviewers was good: all three reviewers agreed in 79.6% of cases and 87.0% of remaining disagreements differed by only one ESI point. There was weak/fair inter-rater reliability in individual ESI estimates (k=0.33, P<0.001).

Of 454 calls placed to 911, 200 (44.1%) were determined not to be appropriate for telemedicine at the time of transfer to the hospital-based telemedicine program. An additional 74 callers (16.3%) required an ambulance response after completion of RT by the NP or PA. In total, 274 (60.4%) resulted in an ambulance response despite being diverted to telemedicine. A minority of callers 36 (7.9%) hung up during RT, however, approximately twice as many callers hung-up during the height of the pandemic (10%) versus the latter period once case counts were decreasing (4.9%, P=0.042).

Of the 180 callers completing remote triage, 77 (17.0%) had their complaints addressed via the RT provider and did not require an ambulance response nor complete a telemedicine visit. 39 (8.6%) patients eligible for a virtual urgent care visit successfully completed a telemedicine encounter: although uncommon, telemedicine visits occurred more frequently during the height of the pandemic (14.3% vs. 2.7%, P<0.001). Conversely, ED referrals via personal transport (5.1%) were more common during the end of the first COVID wave in NYC (8.5% vs. 1.7%, P<0.001).

Ambulance dispatches after transfer to hospital-based telemedicine services were categorized as either due to medical necessity or patient preference: 188 (68.6%) were due to patient preference while 69 (25.2%) were due to medical necessity. During the initial pandemic wave, 129 patients required an ambulance with 49.6% due to patient preference and 41.1% due to medical necessity (Fig. 3). From June until October, when COVID-19 case counts were decreasing across the city, 145 patients required an ambulance with the majority (85.5%) due to patient preference (OR, 6.8; 95% confidence interval, 3.65–13.38). A higher likelihood of ambulance dispatch due to patient preference was observed after adjusting for caller characteristics including age, sex, and borough during the second study period (OR, 6.2; 95% confidence interval, 2.41–18.27).

Follow-up phone calls were performed during the first three months after launch of the intervention. Among 315 (69.4%) attempted calls, we successfully followed up with 31.3% of patients. Nearly two-thirds of patients received medical care in an ED or urgent care after their initial 911 call. Approximately one-quarter (26.1%) were admitted, with one patient requiring intensive care unit level care (this patient was deemed not suitable for telemedicine after RT). Twenty (14.1%) patients for whom an ambulance was dispatched were evaluated by EMS in the field, but ultimately were not transported to the hospital. A small proportion of patients (7.8%) diverted to telemedicine called 911 back immediately, while 12 (8.5%) patients went to the ED on their own. Of the group of patients who called 911 immediately or went to the ED on their own, 18 (78.2%) were admitted. Nine patients (6.3%) contacted their primary care provider and 11 of the patients who did not seek medical care after the call indicated they completed a telemedicine visit.

There are a few notable limitations of our study. All caller information was obtained by self-report during the RT evaluation, which many callers did not complete. Estimates of patient acuity were limited by the lack of available vital signs typically used in the determination of ESI level. However, multiple reviewers were used to determine ESI and the primary interest of our study was to compare ESI during the early and later phases of the pandemic such that any limitations should be consistent across the two time periods.

We were not able to view patient health records or systematically assess outcomes after the diversion to RT and/or telemedicine. Our follow-up with patients was also incomplete, with less than one-third successfully contacted for follow-up. Furthermore, this cohort is limited to callers diverted to our hospital-based telemedicine program and call type and outcomes may have differed at different institutions. However, this is the first study to our knowledge to attempt to contact patients after 911 diversion to telemedicine. Finally, our program was implemented during the height of the COVID-19 pandemic, and while our study period extends to the tail-end of the initial wave, this context significantly limits the generalizability of our study results.