There are two essential and costly components in a typical WSI scanner: the capturing unit, typically a set of lenses with a distortion-calibrated digital eyepiece; and on-board or external high-performance computers. Like any dedicated devices, since both parts are specifically built for the system, they are mostly nonproductive when the system is idle and thus waste much of their value when underused. Unfortunately, this is commonly the case for smaller hospitals in which complicated pathological diagnoses occur only occasionally. This situation, coupled with consumer electronics' performance approaching that of medical-grade tools, led to the idea of creating sWSI with the structure illustrated in Figure 1.

To assess the diagnosis-readiness of sWSI VSs of challenging cases, a clinical evaluation experiment was carried out in Pathology Center, Shanghai General Hospital, School of Medicine, Shanghai Jiaotong University (SJTU-PC) between February 23rd, 2017 and April 15th, 2017. A total of 100 frozen section slides (from one of the five categories listed in Table 3) gathered among inpatients between February 23^rd, 2017 and April 1st, 2017 were randomly selected as the test dataset. The slides varied significantly in size and shape, were prepared routinely by technicians in the department, and may have had common issues with frozen sections, such as unevenness and folding. Consideration of both possibilities of tumors as well as benign lesions were included in the diagnosis, following generally accepted practice standards [30-32].

All samples went through three diagnosis procedures using optical microscopy, sWSI VSs, and high-end standalone scanner VSs under a 20X objective lens. In each procedure, each sample was examined by one or two pathologists independently. These diagnoses were then compared against each other to obtain agreement statistics. The full procedure is illustrated in Figure 7. This would be the first of a planned sequence of experiments evaluating sWSI on a wide range of samples, so we focused on using iPhones (on which the international version of sWSI is supported). The smartphones used were: one iPhone 5s, one iPhone 6, and one iPhone 6 plus, purchased from the second-hand market between US $120 and $200. The microscopes for sWSI scanning were: an Olympus BH2-BHS, an Olympus CX21, and a Phoenix PH50-1B43L-PL, all being low-end bio-microscopes. The scanner used in this experiment was Aperio AT2 from Leica, which is a high-end model. The microscope for optical microscopy was an Olympus BX51, which is another high-end model. Pathologists A, B, C and D are all senior faculty members from SJTU-PC, while the work of pathologist E was carried out by 10 senior and respected pathologists invited from top hospitals across China, as listed in Acknowledgements.

As part of the evaluation, quantifying the learning curve of manual scanning with sWSI was covered in this experiment. The VS quality depends on the sample being in focus and even the scan speed in the scanning process, so it is positively linked to the level of skill of operator or inversely to the time consumed. Presumably, as the operator gains experience, they would scan faster while maintaining the same quality. An intuitive way to measure this assumption is by comparing the time consumption of carefully scanning a unit area of samples under a fixed objective lens magnification against the number of slides scanned by operator. To this end, a total of 15 interns, medical school students, and assistant technicians who were well trained to use microscopes but had no prior experience with sWSI volunteered to scan the slides with sWSI. With duplication, each participant scanned 10 of the 100 slides after watching a demonstration, with the approximate sample size and time consumption recorded. Only one of the duplicated scans of the same sample was randomly selected for accuracy evaluation.

The accuracy of diagnosis in this evaluation experiment was measured on a slide-wise level, since the classifications of regions in the sample can be subjective and pathologists usually consolidate multiple patterns across the whole sample into a conclusion. The sensitivity is defined as whether a pathologist can use a VS to correctly identify all critical regions of interest across the sample. The specificity is defined as whether the pathologist can correctly analyze each region of interest and identify patterns based on a VS.

To represent the performance based on the sensitivity and specificity defined above, each diagnosis was classified into one of three categories: accurate, low sensitivity (LSen), or low specificity (LSpe). For each VS, if the pathologist missed any critical region of interest, the VS was deemed as LSen. With the VS past the sensitivity check, if the pathologist then correctly identified patterns of all regions of interest, it was classified as being accurate; otherwise it was LSpe. When averaged, these metrics were weighted by their sample counts.

Using optical microscopy as the ground truth, the sWSI system achieved good performance similar to that of the standalone scanner in most sample categories except lung, as summarized in Table 4,Table 5,Table 6, and Table 7. For easier comparison, the per-category metrics are plotted in Figure 8, Figure 9, Figure 10, and Figure 11, with the average in Figure 12. Screen shots of VS regions where sWSI provided adequate or poor image quality are illustrated in Figure 13 and Figure 14. Although the performance was not ideal in absolute terms, all pathologists involved in this experiment firmly agreed that sWSI performed very well for these frozen section samples after viewing the results in retrospect. The pathologists also suggested that sWSI is clinically reliable for daily use on easier and more common samples, such as margins or paraffin sections, which are not covered in this experiment.

For validation purposes, all VSs in this experiment captured by sWSI and the scanner can be viewed online through the links provided in Multimedia Appendix 1. Details of each sample and the diagnosis, including the ground truth and those made based on standalone scanners or sWSI, are provided in Multimedia Appendix 2. Limited by time, this record is mostly in its original language (Chinese) but may be translated and requested from the authors.

The learning curve, which is inversely approximated by the scan time normalized for a 15-by-15 mm sample being plotted against the number of samples scanned, is shown in Figure 15. In the experiment, productivity varied significantly among operators, as the slower ones consumed up to 50% more time at the beginning of the experiment. However, the slower operators caught up swiftly through practice and eventually reached just over 35 minutes for every 15-by-15 mm sample, which is comparable to automatic scanners. Most operators reached a stable level of proficiency after scanning just 5 slides. It should be noted that the 20X objective lens for frozen section samples is a relatively challenging application, as the samples are often uneven and require frequent focus adjustments. As a result, the scan speed in Figure 15 may be considered as a worst-case scenario.

First, a significant performance gap exists among different sample categories that are challenging and require very high image quality. Based on the experimental results, sWSI diagnoses of leiomyoma of the uterus, adenomyosis, papillary carcinoma of the thyroid, invasive breast carcinoma, and fibroadenoma of the breast are relatively more reliable than standard procedures, but those for follicular carcinoma of the thyroid, intraductal papillary neoplasms of the breast, and lung cancer are much worse. On the positive side, since the quality can be assessed after scanning and before inspection by the pathologists, these mistakes may be largely avoided in practice, especially those of LSpe (which should be analyzed by optical microscopy). However, this finding does reveal the fact that the current version of sWSI may not be suitable to all types of samples, or at least that testing may be required before formally adopting sWSI for applications requiring the highest quality images.

Second, similar to standalone scanners, sWSI is strongly affected by the physical preparation quality of the samples. Common problems include samples being broken into multiple pieces, varying thickness, and poor staining quality.

Finally, sWSI suffers from an inferior setup of hardware and environments, which comes with its focus on cost and scalability, and may make its competition against standalone scanners challenging. One such example is data storage: standalone scanners are mostly designed to work with high-speed wired and local networks. Since sWSI provides wireless connection-based data transmission and Web browser-based viewing, limits on bandwidth may severely affect image quality. Another example is that VSs produced by scanners are displayed on specialized work station monitors with high coverage of red/green/blue color space and 4K resolution, while those of sWSI are reviewed on normal monitors, or even smartphones with small screens.

The design of the experiment can be improved in future work. Based on discussions with participating pathologists, it can be safely assumed that there would be little error in diagnoses made by senior pathologists. However, from experimental results and comments, there are significant differences in the level of confidence that pathologists have regarding dubious cases where the provided VSs alone are not sufficient for a firm decision. In practical procedures, pathologists would undertake further investigation with other techniques, such as paraffin embedding and sectioning, which is more reliable (but far more time consuming) than frozen sectioning. In this experiment, participants were required to make binary decisions when judging thresholds that were different from each other. As a result, using multiple probabilities instead of a firm answer might be a better alternative.

There are a number of technical issues to be solved in future research and development. First, some parameters on Android phones cannot be controlled through publicly available programing interfaces for older Android operating systems. Weakened control may lead to improper configurations, such as a long exposure time causing blur. Second, the openGL driver which offers general-purpose graphic processing unit (GPGPU) computing potential is very difficult to work with and produces unexpected results on many smartphone models for unknown reasons. Preliminary research using GPGPU on iPhones brought a dramatic boost in processing speed over 60%, but certain models behave improperly. Finally, the real-time movement tracking is inaccurate and leads to location mismatch in subsequent mini-map construction. Although this issue does not affect the VS generation, the mis-drawing in the mini-map can be confusing.

Limited by time and development cost, the international version of sWSI is only offered on iOS. The Chinese version (Tai Rui Jing Xia), which has a complicated still-FoV capturing tool and clinical report system, supports both Android and iOS but may not be accessible from abroad. In follow-up studies, both versions will be evaluated and compared. More quantitative metrics may also be covered (eg, the measure of absolute image quality) even though most pathologist users suggest such measurements have very limited implications in clinical settings.

On the clinical side, only the basic scanning function of sWSI was evaluated in this experiment, leaving sWSI's many other useful functions to be tested. These functions include swift scanning at a lower magnification then adding static FoVs at higher magnification to cover both speed and detail, recording z-stacks of thick smears with video clips, and potential application of sWSI in fluorescent or dark-field microscopy. The validation of these functions, as well as experiments with expanded sample sizes, are planned in future work.

In this paper, an ultra-low-cost WSI system named sWSI (with clients hosted on mainstream Android and iOS smartphones) was introduced and analyzed. Compared to automatic scanners and high-end computer-based solutions, this alternative dramatically reduces the setup cost to as low as US $100 per unit with service costs of US $1-$10 per scan. Although sWSI may not replace existing dedicated devices, it could become a reliable alternative that weighs more on cost-effectiveness.

By employing distributed image processing, both robustness and efficiency are covered. Through high performance computing and real-time feedback, user friendliness is optimized with minimal manual input, leaving most interface-kernel coordination (and even image distortion correction) fully automated. Based on clinical evaluation using 100 frozen section samples, sWSI is considered adequate by expert pathologists for making a diagnosis for most sample types. The overall accuracy based on sWSI VSs is slightly poorer than that based on high-end scanners, which is the current solution for digitizing whole slides. This gap is largely attributed to low specificity for samples requiring higher levels of detail (eg, lungs), which means that most of the inadequacy of sWSI comes from relatively low image quality, which can be identified before diagnoses are made.

In the experiment, 15 operators with no prior experience with sWSI learned to use the manual scanning very quickly. User throughput stably reached between 27 and 36 minutes per 15-by-15 mm sample area under a 20X objective lens after scanning 9 slides each, similar to that of mid-to-low-end scanners. In low-frequency usage situations (eg, remote or low-tier hospitals), this level of labor may be considered cost-effective, given the vast financial savings from deploying sWSI instead of scanners.