<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.0 20040830//EN" "http://dtd.nlm.nih.gov/publishing/2.0/journalpublishing.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article" dtd-version="2.0">
  <front>
    <journal-meta>
      <journal-id journal-id-type="publisher-id">JMU</journal-id>
      <journal-id journal-id-type="nlm-ta">JMIR Mhealth Uhealth</journal-id>
      <journal-title>JMIR mHealth and uHealth</journal-title>
      <issn pub-type="epub">2291-5222</issn>
      <publisher>
        <publisher-name>JMIR Publications</publisher-name>
        <publisher-loc>Toronto, Canada</publisher-loc>
      </publisher>
    </journal-meta>
    <article-meta>
      <article-id pub-id-type="publisher-id">v7i8e14056</article-id>
      <article-id pub-id-type="pmid">31471963</article-id>
      <article-id pub-id-type="doi">10.2196/14056</article-id>
      <article-categories>
        <subj-group subj-group-type="heading">
          <subject>Viewpoint</subject>
        </subj-group>
        <subj-group subj-group-type="article-type">
          <subject>Viewpoint</subject>
        </subj-group>
      </article-categories>
      <title-group>
        <article-title>The SMART Framework: Integration of Citizen Science, Community-Based Participatory Research, and Systems Science for Population Health Science in the Digital Age</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="editor">
          <name>
            <surname>Eysenbach</surname>
            <given-names>Gunther</given-names>
          </name>
        </contrib>
      </contrib-group>
      <contrib-group>
        <contrib contrib-type="reviewer">
          <name>
            <surname>Wald</surname>
            <given-names>Dara</given-names>
          </name>
        </contrib>
        <contrib contrib-type="reviewer">
          <name>
            <surname>Leal Neto</surname>
            <given-names>Onicio</given-names>
          </name>
        </contrib>
      </contrib-group>
      <contrib-group>
        <contrib id="contrib1" contrib-type="author" corresp="yes">
          <name name-style="western">
            <surname>Katapally</surname>
            <given-names>Tarun Reddy</given-names>
          </name>
          <degrees>MS, PhD</degrees>
          <xref rid="aff1" ref-type="aff">1</xref>
          <address>
            <institution>Johnson Shoyama Graduate School of Public Policy</institution>
            <institution>University of Regina</institution>
            <addr-line>2155 College Ave</addr-line>
            <addr-line>Regina, SK, S4P4V5</addr-line>
            <country>Canada</country>
            <phone>1 3065854544</phone>
            <email>tarun.katapally@uregina.ca</email>
          </address>
          <ext-link ext-link-type="orcid">http://orcid.org/0000-0001-5765-1435</ext-link>
        </contrib>
      </contrib-group>
      <aff id="aff1">
        <label>1</label>
        <institution>Johnson Shoyama Graduate School of Public Policy</institution>
        <institution>University of Regina</institution>
        <addr-line>Regina, SK</addr-line>
        <country>Canada</country>
      </aff>
      <author-notes>
        <corresp>Corresponding Author: Tarun Reddy Katapally 
        <email>tarun.katapally@uregina.ca</email></corresp>
      </author-notes>
      <pub-date pub-type="collection"><month>08</month><year>2019</year></pub-date>
      <pub-date pub-type="epub">
        <day>30</day>
        <month>08</month>
        <year>2019</year>
      </pub-date>
      <volume>7</volume>
      <issue>8</issue>
      <elocation-id>e14056</elocation-id>
      <history>
        <date date-type="received">
          <day>18</day>
          <month>3</month>
          <year>2019</year>
        </date>
        <date date-type="rev-request">
          <day>28</day>
          <month>4</month>
          <year>2019</year>
        </date>
        <date date-type="rev-recd">
          <day>20</day>
          <month>7</month>
          <year>2019</year>
        </date>
        <date date-type="accepted">
          <day>28</day>
          <month>7</month>
          <year>2019</year>
        </date>
      </history>
      <copyright-statement>©Tarun Reddy Katapally. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 30.08.2019.</copyright-statement>
      <copyright-year>2019</copyright-year>
      <license license-type="open-access" xlink:href="https://creativecommons.org/licenses/by/4.0/">
        <p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.</p>
      </license>
      <self-uri xlink:href="http://mhealth.jmir.org/2019/8/e14056/" xlink:type="simple"/>
      <abstract>
        <p>Citizen science enables citizens to actively contribute to all aspects of the research process, from conceptualization and data collection, to knowledge translation and evaluation. Citizen science is gradually emerging as a pertinent approach in population health research. Given that citizen science has intrinsic links with community-based research, where participatory action drives the research agenda, these two approaches could be integrated to address complex population health issues. Community-based participatory research has a strong record of application across multiple disciplines and sectors to address health inequities. Citizen science can use the structure of community-based participatory research to take local approaches of problem solving to a global scale, because citizen science emerged through individual environmental activism that is not limited by geography. This synergy has significant implications for population health research if combined with systems science, which can offer theoretical and methodological strength to citizen science and community-based participatory research. Systems science applies a holistic perspective to understand the complex mechanisms underlying causal relationships within and between systems, as it goes beyond linear relationships by utilizing big data–driven advanced computational models. However, to truly integrate citizen science, community-based participatory research, and systems science, it is time to realize the power of ubiquitous digital tools, such as smartphones, for connecting us all and providing big data. Smartphones have the potential to not only create equity by providing a voice to disenfranchised citizens but smartphone-based apps also have the reach and power to source big data to inform policies. An imminent challenge in legitimizing citizen science is minimizing bias, which can be achieved by standardizing methods and enhancing data quality—a rigorous process that requires researchers to collaborate with citizen scientists utilizing the principles of community-based participatory research action. This study advances SMART, an evidence-based framework that integrates citizen science, community-based participatory research, and systems science through ubiquitous tools by addressing core challenges such as citizen engagement, data management, and internet inequity to legitimize this integration.</p>
      </abstract>
      <kwd-group>
        <kwd>community-based participatory research</kwd>
        <kwd>smartphones</kwd>
        <kwd>mobile phones</kwd>
        <kwd>population health</kwd>
        <kwd>mHealth</kwd>
        <kwd>eHealth</kwd>
        <kwd>digital health</kwd>
        <kwd>big data</kwd>
        <kwd>evidence-based framework</kwd>
        <kwd>citizen science</kwd>
        <kwd>participatory research</kwd>
        <kwd>participatory surveillance</kwd>
        <kwd>systems science</kwd>
        <kwd>ubiquitous tools</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec>
      <title>Population Health Science in the Digital Age</title>
      <p>Global population health crises in the 21st century are extremely complex, with links to economic disasters [<xref ref-type="bibr" rid="ref1">1</xref>-<xref ref-type="bibr" rid="ref3">3</xref>], warfare [<xref ref-type="bibr" rid="ref3">3</xref>,<xref ref-type="bibr" rid="ref4">4</xref>], and climate change [<xref ref-type="bibr" rid="ref3">3</xref>,<xref ref-type="bibr" rid="ref5">5</xref>]. The digital age offers new opportunities and challenges for population health science to tackle these global crises. For instance, digital tools and technologies are increasingly being used to not only address urgent humanitarian crises [<xref ref-type="bibr" rid="ref6">6</xref>] but also facilitate citizen participation, population health interventions, and knowledge transfer [<xref ref-type="bibr" rid="ref7">7</xref>].</p>
      <p>However, digital technologies are rarely evaluated for their impact on health outcomes [<xref ref-type="bibr" rid="ref6">6</xref>], and perhaps more relevant to population health, citizens, communities, and researchers are seldom involved in designing digital tools [<xref ref-type="bibr" rid="ref7">7</xref>,<xref ref-type="bibr" rid="ref8">8</xref>]. Given that most digital tools are developed for profit, where privacy and confidentiality are major concerns [<xref ref-type="bibr" rid="ref8">8</xref>,<xref ref-type="bibr" rid="ref6">6</xref>], adoption of digital technology in population health science should take special consideration to remedy ethical pitfalls.</p>
      <p>Given that addressing health inequities is one of the primary goals of population health science [<xref ref-type="bibr" rid="ref9">9</xref>-<xref ref-type="bibr" rid="ref11">11</xref>], there is a role for digital tools and technologies in contributing to the reduction of health disparities. Nevertheless, a digitally driven big data approach has the risk of widening existing health disparities if care is not taken to engage diverse populations to realize potential benefits [<xref ref-type="bibr" rid="ref12">12</xref>]. Complex global health crises are exacerbated by health inequities and are difficult to address using traditional research practices, thus citizen science is emerging as a powerful approach that policy makers are increasingly employing in determining population health solutions [<xref ref-type="bibr" rid="ref13">13</xref>,<xref ref-type="bibr" rid="ref14">14</xref>].</p>
      <p>An example of citizen science interventions is the ability to harness technology to inform active, healthy neighborhoods in developing countries [<xref ref-type="bibr" rid="ref15">15</xref>]. This bottom-up approach of involving community members in research can also be an effective way to translate knowledge to a broader audience [<xref ref-type="bibr" rid="ref14">14</xref>,<xref ref-type="bibr" rid="ref16">16</xref>] and has inherent overlap with community-based participatory research, which is known to facilitate community involvement in informing research, policy, and practice [<xref ref-type="bibr" rid="ref17">17</xref>].</p>
      <p>Community-based participatory research has led the crusade of minimizing health disparities [<xref ref-type="bibr" rid="ref18">18</xref>,<xref ref-type="bibr" rid="ref19">19</xref>] by engaging communities equitably in all aspects of the research process [<xref ref-type="bibr" rid="ref20">20</xref>]. However, global population health problems require holistic systems science solutions that enable the examination of complex interconnected factors influencing health outcomes [<xref ref-type="bibr" rid="ref21">21</xref>,<xref ref-type="bibr" rid="ref22">22</xref>]. As systems science has been identified as particularly effective for addressing health inequities [<xref ref-type="bibr" rid="ref22">22</xref>], it is time now to reimagine the implementation of population health science in the digital age by developing a framework that integrates citizen science, community-based participatory research, and systems science (ie, three-pronged approach).</p>
    </sec>
    <sec>
      <title>Citizen Science</title>
      <p>Citizen science enables citizens to actively contribute to all aspects of the research process, from conceptualization and data collection, to knowledge translation and evaluation [<xref ref-type="bibr" rid="ref23">23</xref>,<xref ref-type="bibr" rid="ref24">24</xref>]. Citizen science has its roots in environmental and ecological activism [<xref ref-type="bibr" rid="ref25">25</xref>], where volunteers across the globe can collect data without being restricted by the constraints of traditional academic research [<xref ref-type="bibr" rid="ref26">26</xref>].</p>
      <p>The applicability of citizen science has become increasingly interdisciplinary over the past couple of decades [<xref ref-type="bibr" rid="ref27">27</xref>], which has implications for population health science—a field of science that plays a key role in addressing broad health inequities [<xref ref-type="bibr" rid="ref28">28</xref>]. Moreover, with increasing power of citizens to effect change [<xref ref-type="bibr" rid="ref13">13</xref>], citizen science is earning a place in national science policies of countries such as the United States and Australia by complementing the efforts of governments and health professionals [<xref ref-type="bibr" rid="ref14">14</xref>].</p>
      <p>Citizen scientists can support and encourage regional data collection in a manner that will help researchers and policy makers to avoid the <italic>one-size fits all</italic> approach [<xref ref-type="bibr" rid="ref14">14</xref>]. Involvement of citizens in research procedures is also an effective way to communicate health information to a broad audience [<xref ref-type="bibr" rid="ref14">14</xref>,<xref ref-type="bibr" rid="ref16">16</xref>]. Rowbotham et al [<xref ref-type="bibr" rid="ref14">14</xref>] argue that citizen science offers an opportunity to transform population health science by engaging a larger proportion of the population in data collection to bring citizen scientist perspectives closer to traditional decision-making processes.</p>
      <p>As citizen science can range from contributory (ie, data collection) and collaborative approaches (ie, analysis and interpretation of data) to cocreation of knowledge (ie, conceptualizing research and translating knowledge) [<xref ref-type="bibr" rid="ref27">27</xref>], it has a natural overlap with community-based participatory research.</p>
    </sec>
    <sec>
      <title>Community-Based Participatory Research and Citizen Science</title>
      <p>Community-based participatory research has a strong record of application across multiple disciplines and sectors to address health inequities [<xref ref-type="bibr" rid="ref29">29</xref>-<xref ref-type="bibr" rid="ref31">31</xref>]. A significant emphasis of community-based participatory research is on generating empirical evidence on social determinants of health, which in essence could be used to inform and influence population health policies.</p>
      <p>However, as community-based participatory research is entrenched in human rights and social justice, it can be applied to promote local and regional policy change by bringing together community needs, scientific evidence, and political power [<xref ref-type="bibr" rid="ref31">31</xref>]. In the path toward policy advocacy, participatory action can be the catalyst in engaging evidence with political power [<xref ref-type="bibr" rid="ref31">31</xref>]. A key component in this participatory action is the involvement of diverse stakeholders in all aspects of the research process through shared responsibility and research ownership [<xref ref-type="bibr" rid="ref29">29</xref>,<xref ref-type="bibr" rid="ref32">32</xref>-<xref ref-type="bibr" rid="ref37">37</xref>].</p>
      <p>This foundational strength of community-based participatory research can result in symbiosis with citizen science in the realm of population health science, as there is considerable alignment between these 2 research approaches [<xref ref-type="bibr" rid="ref38">38</xref>]. To translate citizen science into community voices that could potentially inform and influence policies [<xref ref-type="bibr" rid="ref24">24</xref>], it is imperative that citizen scientist endeavors are structured using community-based participatory research principles, where citizens co-design studies and cocreate knowledge with researchers by contributing to all aspects of the research process [<xref ref-type="bibr" rid="ref24">24</xref>,<xref ref-type="bibr" rid="ref39">39</xref>].</p>
      <p>As citizens are ultimately potential voters, citizen science can perhaps stimulate renewed impetus among decision makers and catalyze evidence-based policy formulation. The alignment with citizen science can catapult community-based participatory research action from local endeavors to global initiatives because citizen science is not restricted by geography, jurisdictions, or populations, thus enabling local solutions to global problems. Data collection on a global scale is the essence of citizen science, and in the 21st century, the potential for participatory modeling with a systems lens enables systems science via citizen science to solve population health issues [<xref ref-type="bibr" rid="ref40">40</xref>].</p>
    </sec>
    <sec>
      <title>Systems Science, Community-Based Participatory Research, and Citizen Science</title>
      <p>Systems science takes a holistic perspective in understanding complex mechanisms by focusing on causality and going beyond linear correlations to advance data analytics through computational modeling that could inform public health decisions [<xref ref-type="bibr" rid="ref41">41</xref>-<xref ref-type="bibr" rid="ref48">48</xref>]. El-Sayed and Galea [<xref ref-type="bibr" rid="ref46">46</xref>] conclude that a systems science approach can upend traditional analytical tools with novel methods such as machine learning, microsimulation, and social network analysis that do justice to the complexity and dynamism of population health science.</p>
      <p>Systems thinking also includes qualitative mapping and problem structuring of ill-defined issues that enable stakeholders to directly address health disparities [<xref ref-type="bibr" rid="ref47">47</xref>]. A systems science approach for structuring problems aligns with community-based participatory research by taking into consideration the need for stakeholder involvement to tackle systemic health inequities [<xref ref-type="bibr" rid="ref48">48</xref>-<xref ref-type="bibr" rid="ref50">50</xref>].</p>
      <p>Frerichs et al [<xref ref-type="bibr" rid="ref41">41</xref>] identified 5 areas of synergy between systems science and community-based participatory research—paradigmatic, socioecological, capacity building, colearning, and translational. These synergies provide a rationale for integrating systems science and community-based participatory research, with the central concept revolving around qualitative problem-structuring and systems mapping — a process that prioritizes research questions for computational modeling to delineate the complex pathways that influence health disparities.</p>
      <p>Systems science can change the paradigm of population health science when combined with community-based participatory research and citizen science. Apart from their interdisciplinarity, the 3 research approaches have tremendous potential for symbiosis if they are used together for reciprocal benefit. For instance, community-based participatory research-informed qualitative mapping could benefit from citizen perceptions at the population level [<xref ref-type="bibr" rid="ref41">41</xref>], an aspect that is core to citizen science.</p>
      <p>Nevertheless, the crux of citizen science is citizen-driven quantitative data collection [<xref ref-type="bibr" rid="ref51">51</xref>], which has implications for both systems science and population health science. Large groups of citizens could provide individual, environmental, and social big data needed for examining interconnected, nonlinear relationships through agent-based modeling that could inform population health policies [<xref ref-type="bibr" rid="ref24">24</xref>,<xref ref-type="bibr" rid="ref52">52</xref>]. A systems perspective can broaden citizen science as an anchor for population health science, which is increasingly adopting socioecological frameworks [<xref ref-type="bibr" rid="ref53">53</xref>-<xref ref-type="bibr" rid="ref55">55</xref>]. The realization of research conducted using these frameworks requires complex multilevel subjective and objective data that enable linkages across geographic and jurisdictional barriers.</p>
      <p>Although there are examples of citizen science projects combining social and ecological data [<xref ref-type="bibr" rid="ref56">56</xref>-<xref ref-type="bibr" rid="ref59">59</xref>], participation of citizens in population health endeavors (ie, large-scale linkage of individual-level health data with administrative and contextual data) has rarely been tested or implemented. Citizen science has predominantly been used for environmental change or ecological activism, whether it is reconstruction of native landscapes in disaster zones in Iraq after the collapse of Saddam Hussein or the determination of social, environmental, and economic associations in high-risk industries such as coal mining [<xref ref-type="bibr" rid="ref60">60</xref>,<xref ref-type="bibr" rid="ref61">61</xref>].</p>
      <p>The bottom-up approach of citizen science to bring about social and environmental change has a place in population health science if integrated with community-based participatory research [<xref ref-type="bibr" rid="ref51">51</xref>]. Nevertheless, the need for big data will eventually thrust systems science to the forefront in conducting population health science [<xref ref-type="bibr" rid="ref62">62</xref>], and this need can be addressed by leveraging citizen-owned ubiquitous digital tools for large-scale data collection by combining citizen science and community-based participatory research.</p>
    </sec>
    <sec>
      <title>Integration Through Ubiquitous Tools</title>
      <p>The growth of citizen science in recent years can be attributed to the global spread of internet-connected devices [<xref ref-type="bibr" rid="ref63">63</xref>], and among all internet-connected devices, the ubiquitous presence of smartphones is the most significant development for citizen science-enabled population health research. The phenomenal growth of smartphone ownership in both developed and developing countries [<xref ref-type="bibr" rid="ref64">64</xref>] creates an opportunity for the proliferation of citizen science across international borders.</p>
      <p>Methodologically, the technology that powers these devices offers extraordinary research opportunities for population health science to overcome traditional constraints in terms of participant recruitment and retention, data collection and analysis, interventions, and knowledge translation [<xref ref-type="bibr" rid="ref24">24</xref>]. Because smartphones have become a key part of day-to-day life and the primary one-stop communication device at home, work, and on the go [<xref ref-type="bibr" rid="ref24">24</xref>,<xref ref-type="bibr" rid="ref65">65</xref>], researchers can collaborate with citizen scientists to triangulate critical data: qualitative perceptions via audio, video, and photo-enabled ecological momentary assessments; traditional and novel quantitative data via deployment of surveys in real time; and objective data sensed via in-built smartphone sensors (eg, accelerometers, pedometers, and global positioning system) [<xref ref-type="bibr" rid="ref24">24</xref>]. This triangulation allows consistent and longitudinal capture of big data in terms of volume, velocity, variety, and veracity [<xref ref-type="bibr" rid="ref66">66</xref>], which is essential for the development of novel computational models informed by systems science approaches.</p>
      <p>Although such comprehensive endeavors are not commonplace at the moment, initiatives such as the SMART Platform are leading the integration of citizen science, community-based participatory research, and systems science in implementing studies for population health surveillance, integrated knowledge translation, and policy interventions [<xref ref-type="bibr" rid="ref24">24</xref>,<xref ref-type="bibr" rid="ref67">67</xref>]. The success of this three-pronged approach will ultimately depend on academic leadership in engaging citizens and communities to leverage data required to address population health issues that are of concern to citizens, communities, policy makers, and researchers.</p>
      <p>According to some estimates, there will be 6 billion smartphones in circulation by the end of 2020 [<xref ref-type="bibr" rid="ref64">64</xref>]. Our lives may be difficult to imagine without these gadgets, and out of all the internet-connected devices that we use, smartphones are the most versatile and pervasive tools. As there is no indication that we will revert to the days without these devices, the real question is, how can we transform smartphones into effective research tools?</p>
      <p>The three-pronged approach provides a paradigm-changing purpose to these research tools. However, for this approach to come to fruition, academia needs to play a role in empowering citizens through smartphones, especially because leaders in the technology industry do not have a strong record of safeguarding citizen privacy and confidentiality [<xref ref-type="bibr" rid="ref68">68</xref>].</p>
    </sec>
    <sec>
      <title>The SMART Framework</title>
      <p>SMART is an evidence-based framework designed to conduct population surveillance, knowledge translation, and interventions by integrating citizen science, community-based participatory research, and systems science. The framework informs the SMART Platform, which researchers are utilizing to engage with citizen scientists via their smartphones to implement multiple studies with varied study designs (eg, cross-sectional and longitudinal studies to quasi-experimental and community trials) across different jurisdictions within and outside of Canada [<xref ref-type="bibr" rid="ref69">69</xref>]. SMART Platform provides the flexibility to engage with participants (ie, citizen scientists) in real time to capture rich population health data across jurisdictions.</p>
      <p>Studies conducted by the SMART Platform apply mixed-methods approaches to understand not just the incidence and prevalence of health behaviors and outcomes but also where, when, how, and, more importantly <italic>why</italic> health behaviors and outcomes change. Comprehensive population health data collection is achieved by triangulating traditional surveys with ecological momentary assessments deployed via smartphones and mobile sensors. These data can be linked with upstream policy and administrative data and downstream health care utilization data to inform policies across jurisdictions in collaboration with primary stakeholders [<xref ref-type="bibr" rid="ref24">24</xref>].</p>
      <p>The SMART Framework is rooted in continuous and consistent engagement of citizen scientists by empowering them to cocreate knowledge. The three-pronged approach allows collaborative development of research questions through qualitative systems mapping that enables data collection across the life course to inform dynamic modeling [<xref ref-type="bibr" rid="ref41">41</xref>]. The evidence generated by this approach is translated back to all relevant stakeholders, including citizen scientists through the same devices that provide the digital data used in all analyses—smartphones (<xref ref-type="fig" rid="figure1">Figure 1</xref>).</p>
      <p><xref ref-type="fig" rid="figure1">Figure 1</xref> enumerates the key components of the SMART Framework (ie, stakeholder engagement on the left side and data processes on the right side) as well as the direction of the data (&gt;) and evidence (&lt;&lt;) flow in relationship with the key stakeholders. The engagement of key stakeholders in the SMART Framework is encapsulated by the interaction between citizens, communities, researchers, and policy makers [<xref ref-type="bibr" rid="ref51">51</xref>].</p>
      <p>The contribution, collaboration, and cocreation cycle is central to citizen science and community-based participatory research, and the engagement of stakeholders that materializes within this cycle is essential for qualitative systems mapping to determine research questions.</p>
      <p>Researchers have a vital role to play in both empowering citizens to participate in collaboration with communities and to translate evidence back to all the stakeholders, including policy makers.</p>
      <fig id="figure1" position="float">
        <label>Figure 1</label>
        <caption>
          <p>The SMART Framework: integration of citizen science, community-based participatory research, and systems science via ubiquitous tools.</p>
        </caption>
        <graphic xlink:href="mhealth_v7i8e14056_fig1.png" alt-version="no" mimetype="image" position="float" xlink:type="simple"/>
      </fig>
      <p>Thus, researchers occupy a central role that straddles both stakeholder engagement and data processes. The evidence translated will enable evaluation of existing research goals in collaboration with stakeholders to inform future data generation.</p>
      <p>Data processes are driven by collection, synthesis, and analysis, with researchers playing the primary role in enabling these processes. Nonetheless, there is a significant potential for citizen involvement in data processes, depending on the level of engagement—contribution, collaboration, or cocreation [<xref ref-type="bibr" rid="ref51">51</xref>]. These processes are dependent on data management, dynamic modeling, and evidence mobilization to ultimately convert data into evidence and translate it back to all stakeholders through the researchers.</p>
      <p>The infinity symbol is representative of the continuous interplay between stakeholders and constant flow of data and evidence within the framework. Digital data leveraged through ubiquitous devices such as smartphones are central to this endless data generation, which is apparent from our dependence on these devices. The stakeholder interplay and data and evidence flow are facilitated by the three-pronged approach, which in turn expedites population health surveillance, integrated knowledge translation, and interventions—processes that are not only interconnected but also interdependent [<xref ref-type="bibr" rid="ref24">24</xref>]. Overall, the framework enables quick replication of studies in different geographic locations, with an option to centralize or decentralize data collection and storage to follow ethical guidelines for data privacy, anonymity, and security.</p>
      <p>An example of this synergy is evident from one of the initiatives informed by the SMART Framework—SMART Indigenous Youth. This initiative is a longitudinal community trial that engages indigenous youth (aged 13-18 years) and educators in rural and remote areas of Canada as citizen scientists [<xref ref-type="bibr" rid="ref70">70</xref>]. The objective of SMART Indigenous Youth is to improve mental health outcomes among youth by embedding a land-based, culturally appropriate active living intervention into the school curricula.</p>
      <p>As part of this initiative, youth and educator citizen scientists are able to cocreate knowledge through a strong partnership between researchers, schools, and rural and remote communities where these citizen scientists reside. The communities are independent jurisdictional units governed by Chief and Council, who also engage with researchers in concert with schools and citizen scientists. This stakeholder engagement accelerates the <italic>big data</italic> collection (surveillance) needed for the dynamic modeling that generates evidence translated to stakeholders (integrated knowledge translation) to evaluate the community trial (intervention).</p>
      <p>SMART Indigenous Youth is an ideal example of research informed by the SMART Framework, and this investigation would not be possible without the power of citizen-owned smartphones. These ubiquitous tools are the reason for real-time engagement of citizen scientists in rural and remote areas, thus they serve as tools of equity for a population that is among one of the most vulnerable in Canada [<xref ref-type="bibr" rid="ref71">71</xref>,<xref ref-type="bibr" rid="ref72">72</xref>].</p>
    </sec>
    <sec>
      <title>Addressing Challenges</title>
      <sec>
        <title>Citizen Engagement: Recruitment, Retention, and Compliance</title>
        <p>The success of citizen science is dependent on its ability to contribute data, or collaborate and cocreate knowledge with researchers [<xref ref-type="bibr" rid="ref51">51</xref>]. However, for these actions to transpire, citizen scientists need to be engaged and empowered. Engagement and empowerment are directly connected to recruitment, retention, and compliance, which are key challenges for the success of citizen science. Through the SMART Platform [<xref ref-type="bibr" rid="ref69">69</xref>], we are integrating citizen science and community-based participatory research to aid citizen engagement and empowerment. For example, when we recruit youth citizen scientists, we not only engage with schools and school administrators as community partners, but we also form youth citizen scientist councils, where youth interact with researchers and school administrators to drive the research process.</p>
        <p>Currently, the SMART Platform is using mobile technology and wearables within and outside of Canada to run simultaneous surveillance and intervention studies with varied designs (eg, cross-sectional and longitudinal studies to quasi-experimental and community trials) among populations across the life course [<xref ref-type="bibr" rid="ref69">69</xref>,<xref ref-type="bibr" rid="ref70">70</xref>,<xref ref-type="bibr" rid="ref73">73</xref>-<xref ref-type="bibr" rid="ref76">76</xref>]. An important factor in maximizing recruitment, retention, and compliance is to develop strategies that are specific to different studies, cohorts, and demographic groups.</p>
        <p>For instance, effective recruitment and retention of older cohorts (aged &gt;65 years) requires continuous engagement and in-person deployment of mobile apps to minimize social isolation and potential technology anxiety. With respect to marginalized populations such as indigenous youth, apart from in-person group deployment through peer-to-peer interaction, incentives such as free data plans play a key role in the success of mobile health interventions.</p>
        <p>Finally, to improve engagement from a mobile technology and methodological perspective, it is crucial to understand how mobile interfaces can be made more intuitive to reduce the burden of traditional surveys instruments. Deploying and testing ecological momentary assessments and developing replicable methods for objective data derivation are strategies used by the SMART Platform to reduce burden and increase overall compliance [<xref ref-type="bibr" rid="ref69">69</xref>,<xref ref-type="bibr" rid="ref77">77</xref>].</p>
        <p>Moreover, the Platform is devised to trigger questions based on time, location, and movement to understand citizen science compliance. The challenges of citizen engagement need to be addressed by a combination of logistical, technological, and methodological solutions—before, during, and after data collection—that are facilitated by the three-pronged approach of the SMART Framework.</p>
      </sec>
      <sec>
        <title>Legitimization: Resources, Power Balance, and Traditional Conservatism</title>
        <p>The SMART Framework’s three-pronged approach inherently faces challenges with legitimization, with leveraging resources being the primary obstacle, as this approach requires significant support in terms of personnel training, funding, and time [<xref ref-type="bibr" rid="ref41">41</xref>]. Furthermore, as computational modeling processes used by researchers lack transparency, the balance of power between researchers, and communities or citizens could be another valid challenge [<xref ref-type="bibr" rid="ref78">78</xref>].</p>
        <p>In addition, traditional conservatism of academic research could be a larger barrier for this three-pronged approach within population health science, where there is risk of reinforcement and perpetuation of the status quo [<xref ref-type="bibr" rid="ref79">79</xref>]. With transformative change necessitating more than the involvement of multiple stakeholders [<xref ref-type="bibr" rid="ref80">80</xref>,<xref ref-type="bibr" rid="ref81">81</xref>], a need exists for continuous evaluation of the three-pronged approach.</p>
        <p>Rowbotham et al [<xref ref-type="bibr" rid="ref14">14</xref>] state that although citizen science has been neglected by population health science, it has the potential to traverse cross-jurisdictional boundaries to provide new insights in solving population health problems. Moreover, in their study exploring the role of citizen science in transforming population health science, the authors also emphasize the role of community-based participatory research in addressing population health issues [<xref ref-type="bibr" rid="ref14">14</xref>].</p>
        <p>Conducting research using the three-pronged approach requires a long-term vision that cannot be restricted to short-term projects. Existing evidence indicates that citizen science projects, especially those that take a systems perspective, require longevity, and although citizen science projects have been able to obtain private and public funding for longitudinal studies through cross-disciplinary funding initiatives, there is an inherent risk of altering the project goals to meet funding requirements [<xref ref-type="bibr" rid="ref82">82</xref>].</p>
        <p>Utilizing the three-pronged SMART Framework, the SMART Platform has evolved into a digital methodological toolkit that can address broad population health issues ranging from the physical inactivity pandemic and youth mental health, to indigenous health and school health policies [<xref ref-type="bibr" rid="ref69">69</xref>,<xref ref-type="bibr" rid="ref70">70</xref>,<xref ref-type="bibr" rid="ref74">74</xref>-<xref ref-type="bibr" rid="ref76">76</xref>]. In doing so, the Platform has been able to secure funding from multiple private and public sources by aligning its goals with priorities of both funding agencies, and citizens and communities.</p>
        <p>This tactic has triggered the interest of interdisciplinary researchers in citizen science–enabled population health studies, which is important for the success of the integration of citizen science, community-based participatory research, and systems science [<xref ref-type="bibr" rid="ref52">52</xref>]. The integration also allows the transfer of power to citizens and communities, who play an important role in evaluating the implementation of the SMART Platform. In the SMART Framework, researchers empower citizens to play a larger role and challenge traditional conservatism in academia. This ultimately allows population health science to use citizen science as a tool when integrated with community-based participatory research and systems science.</p>
      </sec>
      <sec>
        <title>Data Management: Privacy, Security, and Linkages</title>
        <p>Ethical considerations in terms of data privacy, security, and anonymity are at the forefront of the SMART Framework. Smartphone-enabled citizen scientist personal data provide sophisticated granularity in terms of potential identification of participants in real time through a slew of sensors such a global positioning system [<xref ref-type="bibr" rid="ref83">83</xref>].</p>
        <p>Protecting privacy and anonymity of citizens requires advanced encryption processes, which are embedded into the SMART Platform [<xref ref-type="bibr" rid="ref24">24</xref>]. However, before beginning data collection and storing encrypted data, obtaining informed consent is mandatory [<xref ref-type="bibr" rid="ref77">77</xref>]. As part of the SMART Platform, citizens not only provide informed consent through their smartphones, but they are also able to drop out of studies and request the deletion of their data. Citizens can also review the informed consent through their smartphones at any point during the study.</p>
        <p>Beyond the ability to provide and delete data, citizen scientists in the SMART Platform co-own the data and are able to participate in data visualization, contribute to analysis, and translate knowledge—all aspects that are enabled by the integration of citizen science, community-based participatory research, and systems science. However, citizen science–enabled data need to be linked with administrative, policy, and even health care access and utilization data to address population health issues [<xref ref-type="bibr" rid="ref77">77</xref>].</p>
        <p>Data linkages require digital and logistical infrastructure as well as policy maker support. For citizen science engagement, technological tools that facilitate secure data collection, synthesis, analyses, and dissemination are essential [<xref ref-type="bibr" rid="ref52">52</xref>]. These data processes require front-end human-computer interfaces such as smartphone apps and back-end interfaces that encapsulate data management, modeling, and mobilization, as articulated within the SMART Framework.</p>
        <p>The SMART Platform utilizes sophisticated front-end interfaces, which comprise constantly evolving smartphone apps specific to various projects. This evolution is informed by the contribution of citizen scientists, communities, and policy makers. Although there is a growing movement for open-source back-end data management [<xref ref-type="bibr" rid="ref84">84</xref>,<xref ref-type="bibr" rid="ref85">85</xref>], the SMART Platform functions on a closed back-end database. Nevertheless, the data that are processed as part of this closed back-end are made open to public and interdisciplinary researchers after applying rigorous protocols of data anonymization. These efforts are a part of the strategy to increase interoperability through data linkages with other data sources that maximize the potential benefits of citizen science–enabled research [<xref ref-type="bibr" rid="ref56">56</xref>,<xref ref-type="bibr" rid="ref57">57</xref>,<xref ref-type="bibr" rid="ref86">86</xref>-<xref ref-type="bibr" rid="ref91">91</xref>].</p>
        <p>The ultimate success of such data linkages is dependent on policy maker support, an integral part of SMART Framework to translate data into evidence. The SMART Platform aims to implement this approach by linking upstream behavioral citizen science data with downstream health care utilization data in collaboration with policy makers at multiple levels (local, provincial, and federal).</p>
      </sec>
      <sec>
        <title>Internet Inequity: Sociodemographics, Global Gaps, and Policy Initiatives</title>
        <p>Of all the challenges that the integration of citizen science, community-based participatory research, and systems science has to overcome, the most systemic barrier is internet inequity. Internet inequity could be defined as differential access to the internet based on wealth of a country (high-, low- or middle-income), geographic region (urban, rural, or remote), socioeconomic status, gender, age, or ethnicity [<xref ref-type="bibr" rid="ref12">12</xref>,<xref ref-type="bibr" rid="ref92">92</xref>-<xref ref-type="bibr" rid="ref94">94</xref>].</p>
        <p>If the goal of the three-pronged approach is to address global population health inequities through citizen-owned devices, this integration needs to take into account the fact that there is potential for widening existing health disparities through digital divides that exclude vulnerable groups [<xref ref-type="bibr" rid="ref12">12</xref>].</p>
        <p>Large-scale surveys have shown significant differences in access to the internet across socioeconomic status [<xref ref-type="bibr" rid="ref92">92</xref>,<xref ref-type="bibr" rid="ref93">93</xref>]. Buchi et al [<xref ref-type="bibr" rid="ref92">92</xref>] conducted structural modeling using data from representative surveys from 5 countries (New Zealand, Sweden, Switzerland, United Kingdom, and United States) to show that sociodemographics independently account for 50% of variance in usage, with age as the strongest predictor.</p>
        <p>The Pew Research Centre’s Internet Project conducted a study to explore how smartphone dependence (ie, when one’s only means of accessing the internet is via a smartphone) and smartphone use differ between key demographic groups in the United States. Results showed that minority groups and younger, lower income, and less educated users are more likely to be dependent on smartphones. These findings suggest that smartphones can symbolize both equity and inequity depending on access to the internet [<xref ref-type="bibr" rid="ref93">93</xref>].</p>
        <p>The digital divide is a complex phenomenon as depicted in the study by Hilbert [<xref ref-type="bibr" rid="ref94">94</xref>], who investigated nationally installed bandwidth potential of 172 countries from 1984 to 2014. The results indicated that internet bandwidth divide between high- and low-income countries first increased during the period of study and then decreased to historic lows between 2012 and 2014. Although there is no clear pattern in terms of the bandwidth divide across high- and low-income countries, in general, there are apparent links between bandwidth divide and income divide [<xref ref-type="bibr" rid="ref94">94</xref>].</p>
        <p>Although more individuals have access to global bandwidth than ever before [<xref ref-type="bibr" rid="ref94">94</xref>], some countries are more digitally progressive and can provide the road map for better access to the internet [<xref ref-type="bibr" rid="ref95">95</xref>]. With the United Nations declaring that access to the internet is a human right [<xref ref-type="bibr" rid="ref96">96</xref>] and with a significant proportion (&gt;75%) of people in 26 countries agreeing with this assessment [<xref ref-type="bibr" rid="ref97">97</xref>], policy makers have a moral responsibility to address internet inequity.</p>
        <p>Countries such as Canada are tackling internet inequity by allocating resources and setting national targets such as providing high speed internet to 100% of homes and businesses by 2030. This strategy is especially important in reducing the urban-rural divide in access to the internet [<xref ref-type="bibr" rid="ref98">98</xref>].</p>
        <p>Although it is beyond the purview of academics to address structural and systemic issues related to internet inequity, as part of the SMART Platform, researchers are engaging with policy makers, communities, and citizens to develop bottom-up approaches for providing internet access to all participating citizen scientists. For example, to provide youth citizen scientists in rural and remote areas with internet access, free data packages are being negotiated as incentives, and schools are increasing internet access to youth during and after school hours.</p>
      </sec>
    </sec>
    <sec>
      <title>Conclusions</title>
      <p>Citizen science, community-based participatory research, and systems science need to be integrated in addressing global population health problems. However, for this integration to materialize, it is necessary to repurpose citizen-owned ubiquitous communication devices (ie, smartphones) that have revolutionized the ability to sense, share, and link big data. Smartphones have the reach to not only create equity by empowering disenfranchised citizens, but smartphone-based apps also have the capacity to source big data to inform policies through the voice of the citizens. SMART is an evidenced-based framework that integrates citizen science, community-based participatory research, and systems science through ubiquitous tools by addressing challenges such as citizen engagement, data management, and internet inequity to legitimize this integration.</p>
    </sec>
  </body>
  <back>
    <ack>
      <p>This study was funded by the Canadian Institutes of Health Research’s Project Grant (#153226) and the Saskatchewan Health Research Foundation’s Establishment Grant (3779). The author acknowledges the support of Ms Jasmin Bhawra, University of Waterloo, to the Digital Epidemiology and Population Health Laboratory and the SMART Platform. Ms Bhawra also contributed to the creation of the SMART Framework diagram.</p>
    </ack>
    <fn-group>
      <fn fn-type="conflict">
        <p>None declared.</p>
      </fn>
    </fn-group>
    <ref-list>
      <ref id="ref1">
        <label>1</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Borisch</surname>
              <given-names>B</given-names>
            </name>
          </person-group>
          <article-title>Public health in times of austerity</article-title>
          <source>J Public Health Policy</source>
          <year>2014</year>
          <month>05</month>
          <volume>35</volume>
          <issue>2</issue>
          <fpage>249</fpage>
          <lpage>57</lpage>
          <pub-id pub-id-type="doi">10.1057/jphp.2014.7</pub-id>
          <pub-id pub-id-type="medline">24751785</pub-id>
          <pub-id pub-id-type="pii">jphp20147</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref2">
        <label>2</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>de Vogli</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Gimeno</surname>
              <given-names>D</given-names>
            </name>
          </person-group>
          <article-title>The G20 and the three global crises: what prospects for global health?</article-title>
          <source>J Epidemiol Community Health</source>
          <year>2010</year>
          <month>02</month>
          <volume>64</volume>
          <issue>2</issue>
          <fpage>99</fpage>
          <lpage>100</lpage>
          <pub-id pub-id-type="doi">10.1136/jech.2009.094789</pub-id>
          <pub-id pub-id-type="medline">20056960</pub-id>
          <pub-id pub-id-type="pii">64/2/99</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref3">
        <label>3</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Garcia</surname>
              <given-names>KK</given-names>
            </name>
            <name name-style="western">
              <surname>Gostin</surname>
              <given-names>LO</given-names>
            </name>
          </person-group>
          <article-title>One health, one world—the intersecting legal regimes of trade, climate change, food security, humanitarian crises, and migration</article-title>
          <source>Laws</source>
          <year>2012</year>
          <month>04</month>
          <day>4</day>
          <volume>1</volume>
          <issue>1</issue>
          <fpage>4</fpage>
          <lpage>38</lpage>
          <pub-id pub-id-type="doi">10.3390/laws1010004</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref4">
        <label>4</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Russell</surname>
              <given-names>MC</given-names>
            </name>
            <name name-style="western">
              <surname>Figley</surname>
              <given-names>CR</given-names>
            </name>
          </person-group>
          <article-title>Investigating recurrent generational wartime behavioral health crises: part 2 of a preliminary analysis</article-title>
          <source>Psychol Inj Law</source>
          <year>2015</year>
          <month>05</month>
          <day>23</day>
          <volume>8</volume>
          <issue>2</issue>
          <fpage>132</fpage>
          <lpage>52</lpage>
          <pub-id pub-id-type="doi">10.1007/s12207-015-9223-5</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref5">
        <label>5</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Nickerson</surname>
              <given-names>JW</given-names>
            </name>
          </person-group>
          <article-title>Global health on the edge--the humanitarian tipping point</article-title>
          <source>Can J Public Health</source>
          <year>2013</year>
          <month>03</month>
          <day>6</day>
          <volume>104</volume>
          <issue>3</issue>
          <fpage>e267</fpage>
          <lpage>9</lpage>
          <pub-id pub-id-type="doi">10.17269/cjph.104.3714</pub-id>
          <pub-id pub-id-type="medline">23823894</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref6">
        <label>6</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Mesmar</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Talhouk</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Akik</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Olivier</surname>
              <given-names>P</given-names>
            </name>
            <name name-style="western">
              <surname>Elhajj</surname>
              <given-names>IH</given-names>
            </name>
            <name name-style="western">
              <surname>Elbassuoni</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Armoush</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Kalot</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Balaam</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Germani</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Ghattas</surname>
              <given-names>H</given-names>
            </name>
          </person-group>
          <article-title>The impact of digital technology on health of populations affected by humanitarian crises: recent innovations and current gaps</article-title>
          <source>J Public Health Policy</source>
          <year>2016</year>
          <month>11</month>
          <volume>37</volume>
          <issue>Suppl 2</issue>
          <fpage>167</fpage>
          <lpage>200</lpage>
          <pub-id pub-id-type="doi">10.1057/s41271-016-0040-1</pub-id>
          <pub-id pub-id-type="medline">27899794</pub-id>
          <pub-id pub-id-type="pii">10.1057/s41271-016-0040-1</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref7">
        <label>7</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Graham</surname>
              <given-names>GN</given-names>
            </name>
            <name name-style="western">
              <surname>Ostrowski</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Sabina</surname>
              <given-names>AB</given-names>
            </name>
          </person-group>
          <article-title>Population health-based approaches to utilizing digital technology: a strategy for equity</article-title>
          <source>J Public Health Policy</source>
          <year>2016</year>
          <month>11</month>
          <volume>37</volume>
          <issue>Suppl 2</issue>
          <fpage>154</fpage>
          <lpage>66</lpage>
          <pub-id pub-id-type="doi">10.1057/s41271-016-0012-5</pub-id>
          <pub-id pub-id-type="medline">27899799</pub-id>
          <pub-id pub-id-type="pii">10.1057/s41271-016-0012-5</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref8">
        <label>8</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Chauvin</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Rispel</surname>
              <given-names>L</given-names>
            </name>
          </person-group>
          <article-title>Digital technology, population health, and health equity</article-title>
          <source>J Public Health Policy</source>
          <year>2016</year>
          <month>11</month>
          <volume>37</volume>
          <issue>Suppl 2</issue>
          <fpage>145</fpage>
          <lpage>53</lpage>
          <pub-id pub-id-type="doi">10.1057/s41271-016-0041-0</pub-id>
          <pub-id pub-id-type="medline">27899801</pub-id>
          <pub-id pub-id-type="pii">10.1057/s41271-016-0041-0</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref9">
        <label>9</label>
        <nlm-citation citation-type="web">
          <source>Government of Canada</source>
          <year>2019</year>
          <access-date>2019-07-02</access-date>
          <comment>What is the Population Health Approach? 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.canada.ca/en/public-health/services/health-promotion/population-health/population-health-approach.html">https://www.canada.ca/en/public-health/services/health-promotion/population-health/population-health-approach.html</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref10">
        <label>10</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Kindig</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Stoddart</surname>
              <given-names>G</given-names>
            </name>
          </person-group>
          <article-title>What is population health?</article-title>
          <source>Am J Public Health</source>
          <year>2003</year>
          <month>03</month>
          <volume>93</volume>
          <issue>3</issue>
          <fpage>380</fpage>
          <lpage>3</lpage>
          <pub-id pub-id-type="doi">10.2105/ajph.93.3.380</pub-id>
          <pub-id pub-id-type="medline">12604476</pub-id>
          <pub-id pub-id-type="pmcid">PMC1447747</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref11">
        <label>11</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Andermann</surname>
              <given-names>A</given-names>
            </name>
            <collab>CLEAR Collaboration</collab>
          </person-group>
          <article-title>Taking action on the social determinants of health in clinical practice: a framework for health professionals</article-title>
          <source>Can Med Assoc J</source>
          <year>2016</year>
          <month>12</month>
          <day>6</day>
          <volume>188</volume>
          <issue>17-18</issue>
          <fpage>E474</fpage>
          <lpage>83</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://www.cmaj.ca/cgi/pmidlookup?view=long&amp;pmid=27503870"/>
          </comment>
          <pub-id pub-id-type="doi">10.1503/cmaj.160177</pub-id>
          <pub-id pub-id-type="medline">27503870</pub-id>
          <pub-id pub-id-type="pii">cmaj.160177</pub-id>
          <pub-id pub-id-type="pmcid">PMC5135524</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref12">
        <label>12</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Zhang</surname>
              <given-names>X</given-names>
            </name>
            <name name-style="western">
              <surname>Pérez-Stable</surname>
              <given-names>EJ</given-names>
            </name>
            <name name-style="western">
              <surname>Bourne</surname>
              <given-names>PE</given-names>
            </name>
            <name name-style="western">
              <surname>Peprah</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>Duru</surname>
              <given-names>OK</given-names>
            </name>
            <name name-style="western">
              <surname>Breen</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>Berrigan</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Wood</surname>
              <given-names>F</given-names>
            </name>
            <name name-style="western">
              <surname>Jackson</surname>
              <given-names>JS</given-names>
            </name>
            <name name-style="western">
              <surname>Wong</surname>
              <given-names>DW</given-names>
            </name>
            <name name-style="western">
              <surname>Denny</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <article-title>Big data science: opportunities and challenges to address minority health and health disparities in the 21st century</article-title>
          <source>Ethn Dis</source>
          <year>2017</year>
          <volume>27</volume>
          <issue>2</issue>
          <fpage>95</fpage>
          <lpage>106</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/28439179"/>
          </comment>
          <pub-id pub-id-type="doi">10.18865/ed.27.2.95</pub-id>
          <pub-id pub-id-type="medline">28439179</pub-id>
          <pub-id pub-id-type="pii">ed.27.2.95</pub-id>
          <pub-id pub-id-type="pmcid">PMC5398183</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref13">
        <label>13</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>King</surname>
              <given-names>AC</given-names>
            </name>
            <name name-style="western">
              <surname>Winter</surname>
              <given-names>SJ</given-names>
            </name>
            <name name-style="western">
              <surname>Chrisinger</surname>
              <given-names>BW</given-names>
            </name>
            <name name-style="western">
              <surname>Hua</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Banchoff</surname>
              <given-names>AW</given-names>
            </name>
          </person-group>
          <article-title>Maximizing the promise of citizen science to advance health and prevent disease</article-title>
          <source>Prev Med</source>
          <year>2019</year>
          <month>02</month>
          <volume>119</volume>
          <fpage>44</fpage>
          <lpage>7</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/30593793"/>
          </comment>
          <pub-id pub-id-type="doi">10.1016/j.ypmed.2018.12.016</pub-id>
          <pub-id pub-id-type="medline">30593793</pub-id>
          <pub-id pub-id-type="pii">S0091-7435(18)30395-5</pub-id>
          <pub-id pub-id-type="pmcid">PMC6687391</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref14">
        <label>14</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Rowbotham</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>McKinnon</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Leach</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Lamberts</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Hawe</surname>
              <given-names>P</given-names>
            </name>
          </person-group>
          <article-title>Does citizen science have the capacity to transform population health science?</article-title>
          <source>Crit Public Health</source>
          <year>2017</year>
          <month>11</month>
          <day>9</day>
          <volume>29</volume>
          <issue>1</issue>
          <fpage>118</fpage>
          <lpage>28</lpage>
          <pub-id pub-id-type="doi">10.1080/09581596.2017.1395393</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref15">
        <label>15</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Rosas</surname>
              <given-names>LG</given-names>
            </name>
            <name name-style="western">
              <surname>Salvo</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Winter</surname>
              <given-names>SJ</given-names>
            </name>
            <name name-style="western">
              <surname>Cortes</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Rivera</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Rodriguez</surname>
              <given-names>NM</given-names>
            </name>
            <name name-style="western">
              <surname>King</surname>
              <given-names>AC</given-names>
            </name>
          </person-group>
          <article-title>Harnessing technology and citizen science to support neighborhoods that promote active living in Mexico</article-title>
          <source>J Urban Health</source>
          <year>2016</year>
          <month>12</month>
          <volume>93</volume>
          <issue>6</issue>
          <fpage>953</fpage>
          <lpage>73</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/27752825"/>
          </comment>
          <pub-id pub-id-type="doi">10.1007/s11524-016-0081-6</pub-id>
          <pub-id pub-id-type="medline">27752825</pub-id>
          <pub-id pub-id-type="pii">10.1007/s11524-016-0081-6</pub-id>
          <pub-id pub-id-type="pmcid">PMC5126018</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref16">
        <label>16</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>de Cocker</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Chastin</surname>
              <given-names>SF</given-names>
            </name>
            <name name-style="western">
              <surname>de Bourdeaudhuij</surname>
              <given-names>I</given-names>
            </name>
            <name name-style="western">
              <surname>Imbo</surname>
              <given-names>I</given-names>
            </name>
            <name name-style="western">
              <surname>Stragier</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Cardon</surname>
              <given-names>G</given-names>
            </name>
          </person-group>
          <article-title>Citizen science to communicate about public health messages: the reach of a playful online survey on sitting time and physical activity</article-title>
          <source>Health Commun</source>
          <year>2019</year>
          <month>06</month>
          <volume>34</volume>
          <issue>7</issue>
          <fpage>720</fpage>
          <lpage>5</lpage>
          <pub-id pub-id-type="doi">10.1080/10410236.2018.1433955</pub-id>
          <pub-id pub-id-type="medline">29412005</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref17">
        <label>17</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Rosenthal</surname>
              <given-names>MS</given-names>
            </name>
          </person-group>
          <article-title>What we might accomplish by engaging in our local communities</article-title>
          <source>Prev Med Rep</source>
          <year>2015</year>
          <volume>2</volume>
          <fpage>13</fpage>
          <lpage>4</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://linkinghub.elsevier.com/retrieve/pii/S2211-3355(14)00020-5"/>
          </comment>
          <pub-id pub-id-type="doi">10.1016/j.pmedr.2014.11.007</pub-id>
          <pub-id pub-id-type="medline">26844045</pub-id>
          <pub-id pub-id-type="pii">S2211-3355(14)00020-5</pub-id>
          <pub-id pub-id-type="pmcid">PMC4721452</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref18">
        <label>18</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Thompson</surname>
              <given-names>B</given-names>
            </name>
            <name name-style="western">
              <surname>Molina</surname>
              <given-names>Y</given-names>
            </name>
            <name name-style="western">
              <surname>Viswanath</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Warnecke</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Prelip</surname>
              <given-names>ML</given-names>
            </name>
          </person-group>
          <article-title>Strategies to empower communities to reduce health disparities</article-title>
          <source>Health Aff (Millwood)</source>
          <year>2016</year>
          <month>08</month>
          <day>1</day>
          <volume>35</volume>
          <issue>8</issue>
          <fpage>1424</fpage>
          <lpage>8</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/27503967"/>
          </comment>
          <pub-id pub-id-type="doi">10.1377/hlthaff.2015.1364</pub-id>
          <pub-id pub-id-type="medline">27503967</pub-id>
          <pub-id pub-id-type="pii">35/8/1424</pub-id>
          <pub-id pub-id-type="pmcid">PMC5554943</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref19">
        <label>19</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Kuo</surname>
              <given-names>AK</given-names>
            </name>
            <name name-style="western">
              <surname>Summers</surname>
              <given-names>NM</given-names>
            </name>
            <name name-style="western">
              <surname>Vohra</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Kahn</surname>
              <given-names>RS</given-names>
            </name>
            <name name-style="western">
              <surname>Bibbins-Domingo</surname>
              <given-names>K</given-names>
            </name>
          </person-group>
          <article-title>The promise of precision population health: reducing health disparities through a community partnership framework</article-title>
          <source>Adv Pediatr</source>
          <year>2019</year>
          <month>08</month>
          <volume>66</volume>
          <fpage>1</fpage>
          <lpage>13</lpage>
          <pub-id pub-id-type="doi">10.1016/j.yapd.2019.03.002</pub-id>
          <pub-id pub-id-type="medline">31230687</pub-id>
          <pub-id pub-id-type="pii">S0065-3101(19)30002-7</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref20">
        <label>20</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Cacari-Stone</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Wallerstein</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>Garcia</surname>
              <given-names>AP</given-names>
            </name>
            <name name-style="western">
              <surname>Minkler</surname>
              <given-names>M</given-names>
            </name>
          </person-group>
          <article-title>The promise of community-based participatory research for health equity: a conceptual model for bridging evidence with policy</article-title>
          <source>Am J Public Health</source>
          <year>2014</year>
          <month>09</month>
          <volume>104</volume>
          <issue>9</issue>
          <fpage>1615</fpage>
          <lpage>23</lpage>
          <pub-id pub-id-type="doi">10.2105/AJPH.2014.301961</pub-id>
          <pub-id pub-id-type="medline">25033119</pub-id>
          <pub-id pub-id-type="pmcid">PMC4151933</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref21">
        <label>21</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>BeLue</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Carmack</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Myers</surname>
              <given-names>KR</given-names>
            </name>
            <name name-style="western">
              <surname>Weinreb-Welch</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Lengerich</surname>
              <given-names>EJ</given-names>
            </name>
          </person-group>
          <article-title>Systems thinking tools as applied to community-based participatory research: a case study</article-title>
          <source>Health Educ Behav</source>
          <year>2012</year>
          <month>12</month>
          <volume>39</volume>
          <issue>6</issue>
          <fpage>745</fpage>
          <lpage>51</lpage>
          <pub-id pub-id-type="doi">10.1177/1090198111430708</pub-id>
          <pub-id pub-id-type="medline">22467637</pub-id>
          <pub-id pub-id-type="pii">1090198111430708</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref22">
        <label>22</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Friel</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Pescud</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Malbon</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>Lee</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Carter</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Greenfield</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Cobcroft</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Potter</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Rychetnik</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Meertens</surname>
              <given-names>B</given-names>
            </name>
          </person-group>
          <article-title>Using systems science to understand the determinants of inequities in healthy eating</article-title>
          <source>PLoS One</source>
          <year>2017</year>
          <volume>12</volume>
          <issue>11</issue>
          <fpage>e0188872</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://dx.plos.org/10.1371/journal.pone.0188872"/>
          </comment>
          <pub-id pub-id-type="doi">10.1371/journal.pone.0188872</pub-id>
          <pub-id pub-id-type="medline">29190662</pub-id>
          <pub-id pub-id-type="pii">PONE-D-17-14398</pub-id>
          <pub-id pub-id-type="pmcid">PMC5708780</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref23">
        <label>23</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Silvertown</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <article-title>A new dawn for citizen science</article-title>
          <source>Trends Ecol Evol</source>
          <year>2009</year>
          <month>09</month>
          <volume>24</volume>
          <issue>9</issue>
          <fpage>467</fpage>
          <lpage>71</lpage>
          <pub-id pub-id-type="doi">10.1016/j.tree.2009.03.017</pub-id>
          <pub-id pub-id-type="medline">19586682</pub-id>
          <pub-id pub-id-type="pii">S0169-5347(09)00175-X</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref24">
        <label>24</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Katapally</surname>
              <given-names>TR</given-names>
            </name>
            <name name-style="western">
              <surname>Bhawra</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Leatherdale</surname>
              <given-names>ST</given-names>
            </name>
            <name name-style="western">
              <surname>Ferguson</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Longo</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Rainham</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Larouche</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Osgood</surname>
              <given-names>N</given-names>
            </name>
          </person-group>
          <article-title>The SMART study, a mobile health and citizen science methodological platform for active living surveillance, integrated knowledge translation, and policy interventions: longitudinal study</article-title>
          <source>JMIR Public Health Surveill</source>
          <year>2018</year>
          <month>03</month>
          <day>27</day>
          <volume>4</volume>
          <issue>1</issue>
          <fpage>e31</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://publichealth.jmir.org/2018/1/e31/"/>
          </comment>
          <pub-id pub-id-type="doi">10.2196/publichealth.8953</pub-id>
          <pub-id pub-id-type="medline">29588267</pub-id>
          <pub-id pub-id-type="pii">v4i1e31</pub-id>
          <pub-id pub-id-type="pmcid">PMC5893892</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref25">
        <label>25</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Dickinson</surname>
              <given-names>JL</given-names>
            </name>
            <name name-style="western">
              <surname>Zuckerberg</surname>
              <given-names>B</given-names>
            </name>
            <name name-style="western">
              <surname>Bonter</surname>
              <given-names>DN</given-names>
            </name>
          </person-group>
          <article-title>Citizen science as an ecological research tool: challenges and benefits</article-title>
          <source>Annu Rev Ecol Evol Syst</source>
          <year>2010</year>
          <month>12</month>
          <volume>41</volume>
          <issue>1</issue>
          <fpage>149</fpage>
          <lpage>72</lpage>
          <pub-id pub-id-type="doi">10.1146/annurev-ecolsys-102209-144636</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref26">
        <label>26</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Theobald</surname>
              <given-names>EJ</given-names>
            </name>
            <name name-style="western">
              <surname>Ettinger</surname>
              <given-names>AK</given-names>
            </name>
            <name name-style="western">
              <surname>Burgess</surname>
              <given-names>HK</given-names>
            </name>
            <name name-style="western">
              <surname>DeBey</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Schmidt</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>Froehlich</surname>
              <given-names>H</given-names>
            </name>
            <name name-style="western">
              <surname>Wagner</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>HilleRisLambers</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Tewksbury</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Harsch</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Parrish</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <article-title>Global change and local solutions: tapping the unrealized potential of citizen science for biodiversity research</article-title>
          <source>Biol Cons</source>
          <year>2015</year>
          <month>01</month>
          <volume>181</volume>
          <fpage>236</fpage>
          <lpage>44</lpage>
          <pub-id pub-id-type="doi">10.1016/j.biocon.2014.10.021</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref27">
        <label>27</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Bonney</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Cooper</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Ballard</surname>
              <given-names>H</given-names>
            </name>
          </person-group>
          <article-title>The theory and practice of citizen science: launching a new journal</article-title>
          <source>Citiz Sci: Theory Practice</source>
          <year>2016</year>
          <month>05</month>
          <day>20</day>
          <volume>1</volume>
          <issue>1</issue>
          <fpage>1</fpage>
          <pub-id pub-id-type="doi">10.5334/cstp.65</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref28">
        <label>28</label>
        <nlm-citation citation-type="book">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Farmer</surname>
              <given-names>RD</given-names>
            </name>
            <name name-style="western">
              <surname>Lawrenson</surname>
              <given-names>R</given-names>
            </name>
          </person-group>
          <source>Lecture Notes: Epidemiology and Public Health Medicine</source>
          <year>2004</year>
          <publisher-loc>Massachusetts</publisher-loc>
          <publisher-name>Wiley-Blackwell</publisher-name>
        </nlm-citation>
      </ref>
      <ref id="ref29">
        <label>29</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Israel</surname>
              <given-names>BA</given-names>
            </name>
            <name name-style="western">
              <surname>Coombe</surname>
              <given-names>CM</given-names>
            </name>
            <name name-style="western">
              <surname>Cheezum</surname>
              <given-names>RR</given-names>
            </name>
            <name name-style="western">
              <surname>Schulz</surname>
              <given-names>AJ</given-names>
            </name>
            <name name-style="western">
              <surname>McGranaghan</surname>
              <given-names>RJ</given-names>
            </name>
            <name name-style="western">
              <surname>Lichtenstein</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Reyes</surname>
              <given-names>AG</given-names>
            </name>
            <name name-style="western">
              <surname>Clement</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Burris</surname>
              <given-names>A</given-names>
            </name>
          </person-group>
          <article-title>Community-based participatory research: a capacity-building approach for policy advocacy aimed at eliminating health disparities</article-title>
          <source>Am J Public Health</source>
          <year>2010</year>
          <month>11</month>
          <volume>100</volume>
          <issue>11</issue>
          <fpage>2094</fpage>
          <lpage>102</lpage>
          <pub-id pub-id-type="doi">10.2105/AJPH.2009.170506</pub-id>
          <pub-id pub-id-type="medline">20864728</pub-id>
          <pub-id pub-id-type="pii">AJPH.2009.170506</pub-id>
          <pub-id pub-id-type="pmcid">PMC2951933</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref30">
        <label>30</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Carroll-Scott</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Toy</surname>
              <given-names>P</given-names>
            </name>
            <name name-style="western">
              <surname>Wyn</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Zane</surname>
              <given-names>JI</given-names>
            </name>
            <name name-style="western">
              <surname>Wallace</surname>
              <given-names>SP</given-names>
            </name>
          </person-group>
          <article-title>Results from the data &amp; democracy initiative to enhance community-based organization data and research capacity</article-title>
          <source>Am J Public Health</source>
          <year>2012</year>
          <month>07</month>
          <volume>102</volume>
          <issue>7</issue>
          <fpage>1384</fpage>
          <lpage>91</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/22594748"/>
          </comment>
          <pub-id pub-id-type="doi">10.2105/AJPH.2011.300457</pub-id>
          <pub-id pub-id-type="medline">22594748</pub-id>
          <pub-id pub-id-type="pmcid">PMC3478024</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref31">
        <label>31</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Freudenberg</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>Tsui</surname>
              <given-names>E</given-names>
            </name>
          </person-group>
          <article-title>Evidence, power, and policy change in community-based participatory research</article-title>
          <source>Am J Public Health</source>
          <year>2014</year>
          <month>01</month>
          <volume>104</volume>
          <issue>1</issue>
          <fpage>11</fpage>
          <lpage>4</lpage>
          <pub-id pub-id-type="doi">10.2105/AJPH.2013.301471</pub-id>
          <pub-id pub-id-type="medline">24228677</pub-id>
          <pub-id pub-id-type="pmcid">PMC3910045</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref32">
        <label>32</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Minkler</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Vasquez</surname>
              <given-names>VB</given-names>
            </name>
            <name name-style="western">
              <surname>Chang</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Miller</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Rubin</surname>
              <given-names>V</given-names>
            </name>
            <name name-style="western">
              <surname>Blackwell</surname>
              <given-names>AG</given-names>
            </name>
            <name name-style="western">
              <surname>Thompson</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Flournoy</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Bell</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <source>University of Washington</source>
          <year>2008</year>
          <access-date>2019-08-21</access-date>
          <comment>Promoting Healthy Public Policy through Community-Based Participatory Research: Ten Case Studies 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://depts.washington.edu/ccph/pdf_files/CBPR_final.pdf">https://depts.washington.edu/ccph/pdf_files/CBPR_final.pdf</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref33">
        <label>33</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Minkler</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Vásquez</surname>
              <given-names>VB</given-names>
            </name>
            <name name-style="western">
              <surname>Tajik</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Petersen</surname>
              <given-names>D</given-names>
            </name>
          </person-group>
          <article-title>Promoting environmental justice through community-based participatory research: the role of community and partnership capacity</article-title>
          <source>Health Educ Behav</source>
          <year>2008</year>
          <month>02</month>
          <volume>35</volume>
          <issue>1</issue>
          <fpage>119</fpage>
          <lpage>37</lpage>
          <pub-id pub-id-type="doi">10.1177/1090198106287692</pub-id>
          <pub-id pub-id-type="medline">16861594</pub-id>
          <pub-id pub-id-type="pii">1090198106287692</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref34">
        <label>34</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Vásquez</surname>
              <given-names>VB</given-names>
            </name>
            <name name-style="western">
              <surname>Minkler</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Shepard</surname>
              <given-names>P</given-names>
            </name>
          </person-group>
          <article-title>Promoting environmental health policy through community based participatory research: a case study from Harlem, New York</article-title>
          <source>J Urban Health</source>
          <year>2006</year>
          <month>01</month>
          <volume>83</volume>
          <issue>1</issue>
          <fpage>101</fpage>
          <lpage>10</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/16736358"/>
          </comment>
          <pub-id pub-id-type="doi">10.1007/s11524-005-9010-9</pub-id>
          <pub-id pub-id-type="medline">16736358</pub-id>
          <pub-id pub-id-type="pmcid">PMC2258322</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref35">
        <label>35</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Vásquez</surname>
              <given-names>VB</given-names>
            </name>
            <name name-style="western">
              <surname>Lanza</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Hennessey-Lavery</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Facente</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Halpin</surname>
              <given-names>HA</given-names>
            </name>
            <name name-style="western">
              <surname>Minkler</surname>
              <given-names>M</given-names>
            </name>
          </person-group>
          <article-title>Addressing food security through public policy action in a community-based participatory research partnership</article-title>
          <source>Health Promot Pract</source>
          <year>2007</year>
          <month>10</month>
          <volume>8</volume>
          <issue>4</issue>
          <fpage>342</fpage>
          <lpage>9</lpage>
          <pub-id pub-id-type="doi">10.1177/1524839906298501</pub-id>
          <pub-id pub-id-type="medline">17728199</pub-id>
          <pub-id pub-id-type="pii">1524839906298501</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref36">
        <label>36</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Petersen</surname>
              <given-names>DM</given-names>
            </name>
            <name name-style="western">
              <surname>Minkler</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Vásquez</surname>
              <given-names>VB</given-names>
            </name>
            <name name-style="western">
              <surname>Kegler</surname>
              <given-names>MC</given-names>
            </name>
            <name name-style="western">
              <surname>Malcoe</surname>
              <given-names>LH</given-names>
            </name>
            <name name-style="western">
              <surname>Whitecrow</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <article-title>Using community-based participatory research to shape policy and prevent lead exposure among native American children</article-title>
          <source>Prog Community Health Partnersh</source>
          <year>2007</year>
          <volume>1</volume>
          <issue>3</issue>
          <fpage>249</fpage>
          <lpage>56</lpage>
          <pub-id pub-id-type="doi">10.1353/cpr.2007.0028</pub-id>
          <pub-id pub-id-type="medline">20208287</pub-id>
          <pub-id pub-id-type="pii">S1557055X07302490</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref37">
        <label>37</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Tajik</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Minkler</surname>
              <given-names>M</given-names>
            </name>
          </person-group>
          <article-title>Environmental justice research and action: a case study in political economy and community-academic collaboration</article-title>
          <source>Int Q Community Health Educ</source>
          <year>2006</year>
          <volume>26</volume>
          <issue>3</issue>
          <fpage>213</fpage>
          <lpage>31</lpage>
          <pub-id pub-id-type="doi">10.2190/IQ.26.3.b</pub-id>
          <pub-id pub-id-type="medline">17827092</pub-id>
          <pub-id pub-id-type="pii">167W5R110136782L</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref38">
        <label>38</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Gregory</surname>
              <given-names>AJ</given-names>
            </name>
            <name name-style="western">
              <surname>Atkins</surname>
              <given-names>JP</given-names>
            </name>
          </person-group>
          <article-title>Community operational research and citizen science: two icons in need of each other?</article-title>
          <source>Eur J Oper Res</source>
          <year>2018</year>
          <month>08</month>
          <volume>268</volume>
          <issue>3</issue>
          <fpage>1111</fpage>
          <lpage>24</lpage>
          <pub-id pub-id-type="doi">10.1016/j.ejor.2017.07.037</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref39">
        <label>39</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hinckson</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>Schneider</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Winter</surname>
              <given-names>SJ</given-names>
            </name>
            <name name-style="western">
              <surname>Stone</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>Puhan</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Stathi</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Porter</surname>
              <given-names>MM</given-names>
            </name>
            <name name-style="western">
              <surname>Gardiner</surname>
              <given-names>PA</given-names>
            </name>
            <name name-style="western">
              <surname>dos Santos</surname>
              <given-names>DL</given-names>
            </name>
            <name name-style="western">
              <surname>Wolff</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>King</surname>
              <given-names>AC</given-names>
            </name>
          </person-group>
          <article-title>Citizen science applied to building healthier community environments: advancing the field through shared construct and measurement development</article-title>
          <source>Int J Behav Nutr Phys Act</source>
          <year>2017</year>
          <month>09</month>
          <day>29</day>
          <volume>14</volume>
          <issue>1</issue>
          <fpage>133</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://ijbnpa.biomedcentral.com/articles/10.1186/s12966-017-0588-6"/>
          </comment>
          <pub-id pub-id-type="doi">10.1186/s12966-017-0588-6</pub-id>
          <pub-id pub-id-type="medline">28962580</pub-id>
          <pub-id pub-id-type="pii">10.1186/s12966-017-0588-6</pub-id>
          <pub-id pub-id-type="pmcid">PMC5622546</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref40">
        <label>40</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Gray</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Jordan</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Crall</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Newman</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Hmelo-Silver</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Huang</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Novak</surname>
              <given-names>W</given-names>
            </name>
            <name name-style="western">
              <surname>Mellor</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Frensley</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Prysby</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Singer</surname>
              <given-names>A</given-names>
            </name>
          </person-group>
          <article-title>Combining participatory modelling and citizen science to support volunteer conservation action</article-title>
          <source>Biol Cons</source>
          <year>2017</year>
          <month>04</month>
          <volume>208</volume>
          <fpage>76</fpage>
          <lpage>86</lpage>
          <pub-id pub-id-type="doi">10.1016/j.biocon.2016.07.037</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref41">
        <label>41</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Frerichs</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Lich</surname>
              <given-names>KH</given-names>
            </name>
            <name name-style="western">
              <surname>Dave</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Corbie-Smith</surname>
              <given-names>G</given-names>
            </name>
          </person-group>
          <article-title>Integrating systems science and community-based participatory research to achieve health equity</article-title>
          <source>Am J Public Health</source>
          <year>2016</year>
          <month>02</month>
          <volume>106</volume>
          <issue>2</issue>
          <fpage>215</fpage>
          <lpage>22</lpage>
          <pub-id pub-id-type="doi">10.2105/AJPH.2015.302944</pub-id>
          <pub-id pub-id-type="medline">26691110</pub-id>
          <pub-id pub-id-type="pmcid">PMC4815818</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref42">
        <label>42</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Mabry</surname>
              <given-names>PL</given-names>
            </name>
            <name name-style="western">
              <surname>Olster</surname>
              <given-names>DH</given-names>
            </name>
            <name name-style="western">
              <surname>Morgan</surname>
              <given-names>GD</given-names>
            </name>
            <name name-style="western">
              <surname>Abrams</surname>
              <given-names>DB</given-names>
            </name>
          </person-group>
          <article-title>Interdisciplinarity and systems science to improve population health: a view from the NIH Office of Behavioral and Social Sciences Research</article-title>
          <source>Am J Prev Med</source>
          <year>2008</year>
          <month>08</month>
          <volume>35</volume>
          <issue>2 Suppl</issue>
          <fpage>S211</fpage>
          <lpage>24</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/18619402"/>
          </comment>
          <pub-id pub-id-type="doi">10.1016/j.amepre.2008.05.018</pub-id>
          <pub-id pub-id-type="medline">18619402</pub-id>
          <pub-id pub-id-type="pii">S0749-3797(08)00431-5</pub-id>
          <pub-id pub-id-type="pmcid">PMC2587290</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref43">
        <label>43</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hammond</surname>
              <given-names>RA</given-names>
            </name>
          </person-group>
          <article-title>Complex systems modeling for obesity research</article-title>
          <source>Prev Chronic Dis</source>
          <year>2009</year>
          <month>07</month>
          <volume>6</volume>
          <issue>3</issue>
          <fpage>A97</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.cdc.gov/pcd/issues/2009/Jul/09_0017.htm"/>
          </comment>
          <pub-id pub-id-type="medline">19527598</pub-id>
          <pub-id pub-id-type="pii">A96</pub-id>
          <pub-id pub-id-type="pmcid">PMC2722404</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref44">
        <label>44</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Leischow</surname>
              <given-names>SJ</given-names>
            </name>
            <name name-style="western">
              <surname>Best</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Trochim</surname>
              <given-names>WM</given-names>
            </name>
            <name name-style="western">
              <surname>Clark</surname>
              <given-names>PI</given-names>
            </name>
            <name name-style="western">
              <surname>Gallagher</surname>
              <given-names>RS</given-names>
            </name>
            <name name-style="western">
              <surname>Marcus</surname>
              <given-names>SE</given-names>
            </name>
            <name name-style="western">
              <surname>Matthews</surname>
              <given-names>E</given-names>
            </name>
          </person-group>
          <article-title>Systems thinking to improve the public's health</article-title>
          <source>Am J Prev Med</source>
          <year>2008</year>
          <month>08</month>
          <volume>35</volume>
          <issue>2 Suppl</issue>
          <fpage>S196</fpage>
          <lpage>203</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/18619400"/>
          </comment>
          <pub-id pub-id-type="doi">10.1016/j.amepre.2008.05.014</pub-id>
          <pub-id pub-id-type="medline">18619400</pub-id>
          <pub-id pub-id-type="pii">S0749-3797(08)00425-X</pub-id>
          <pub-id pub-id-type="pmcid">PMC3940421</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref45">
        <label>45</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Luke</surname>
              <given-names>DA</given-names>
            </name>
            <name name-style="western">
              <surname>Stamatakis</surname>
              <given-names>KA</given-names>
            </name>
          </person-group>
          <article-title>Systems science methods in public health: dynamics, networks, and agents</article-title>
          <source>Annu Rev Public Health</source>
          <year>2012</year>
          <month>04</month>
          <volume>33</volume>
          <fpage>357</fpage>
          <lpage>76</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/22224885"/>
          </comment>
          <pub-id pub-id-type="doi">10.1146/annurev-publhealth-031210-101222</pub-id>
          <pub-id pub-id-type="medline">22224885</pub-id>
          <pub-id pub-id-type="pmcid">PMC3644212</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref46">
        <label>46</label>
        <nlm-citation citation-type="book">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>El-Sayed</surname>
              <given-names>AM</given-names>
            </name>
            <name name-style="western">
              <surname>Galea</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <source>Systems Science and Population Health</source>
          <year>2017</year>
          <publisher-loc>New York</publisher-loc>
          <publisher-name>Oxford University Press</publisher-name>
        </nlm-citation>
      </ref>
      <ref id="ref47">
        <label>47</label>
        <nlm-citation citation-type="book">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Rosenhead</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Mingers</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <source>Rational Analysis for a Problematic World Revisited: Problem Structuring Methods for Complexity, Uncertainty and Conflict. Second Edition</source>
          <year>2001</year>
          <publisher-loc>Chichester, UK</publisher-loc>
          <publisher-name>Wiley</publisher-name>
        </nlm-citation>
      </ref>
      <ref id="ref48">
        <label>48</label>
        <nlm-citation citation-type="book">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Midgley</surname>
              <given-names>G</given-names>
            </name>
          </person-group>
          <source>Systemic Intervention: Philosophy, Methodology, and Practice</source>
          <year>2000</year>
          <publisher-loc>New York</publisher-loc>
          <publisher-name>Kluwer Academic/Plenum Publishers</publisher-name>
        </nlm-citation>
      </ref>
      <ref id="ref49">
        <label>49</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Gillen</surname>
              <given-names>EM</given-names>
            </name>
            <name name-style="western">
              <surname>Lich</surname>
              <given-names>KH</given-names>
            </name>
            <name name-style="western">
              <surname>Yeatts</surname>
              <given-names>KB</given-names>
            </name>
            <name name-style="western">
              <surname>Hernandez</surname>
              <given-names>ML</given-names>
            </name>
            <name name-style="western">
              <surname>Smith</surname>
              <given-names>TW</given-names>
            </name>
            <name name-style="western">
              <surname>Lewis</surname>
              <given-names>MA</given-names>
            </name>
          </person-group>
          <article-title>Social ecology of asthma: engaging stakeholders in integrating health behavior theories and practice-based evidence through systems mapping</article-title>
          <source>Health Educ Behav</source>
          <year>2014</year>
          <month>02</month>
          <volume>41</volume>
          <issue>1</issue>
          <fpage>63</fpage>
          <lpage>77</lpage>
          <pub-id pub-id-type="doi">10.1177/1090198113486804</pub-id>
          <pub-id pub-id-type="medline">23709516</pub-id>
          <pub-id pub-id-type="pii">1090198113486804</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref50">
        <label>50</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Kitson</surname>
              <given-names>NA</given-names>
            </name>
            <name name-style="western">
              <surname>Price</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Lau</surname>
              <given-names>FY</given-names>
            </name>
            <name name-style="western">
              <surname>Showler</surname>
              <given-names>G</given-names>
            </name>
          </person-group>
          <article-title>Developing a medication communication framework across continuums of care using the Circle of Care Modeling approach</article-title>
          <source>BMC Health Serv Res</source>
          <year>2013</year>
          <month>10</month>
          <day>17</day>
          <volume>13</volume>
          <fpage>418</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-13-418"/>
          </comment>
          <pub-id pub-id-type="doi">10.1186/1472-6963-13-418</pub-id>
          <pub-id pub-id-type="medline">24134454</pub-id>
          <pub-id pub-id-type="pii">1472-6963-13-418</pub-id>
          <pub-id pub-id-type="pmcid">PMC3853098</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref51">
        <label>51</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Bonney</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Cooper</surname>
              <given-names>CB</given-names>
            </name>
            <name name-style="western">
              <surname>Dickinson</surname>
              <given-names>JL</given-names>
            </name>
            <name name-style="western">
              <surname>Kelling</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Phillips</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Rosenberg</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Shirk</surname>
              <given-names>JL</given-names>
            </name>
          </person-group>
          <article-title>Citizen science: a developing tool for expanding science knowledge and scientific literacy</article-title>
          <source>BioSci</source>
          <year>2009</year>
          <volume>59</volume>
          <issue>11</issue>
          <fpage>977</fpage>
          <lpage>84</lpage>
          <pub-id pub-id-type="doi">10.1525/bio.2009.59.11.9</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref52">
        <label>52</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Crain</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Cooper</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Dickinson</surname>
              <given-names>JL</given-names>
            </name>
          </person-group>
          <article-title>Citizen science: a tool for integrating studies of human and natural systems</article-title>
          <source>Annu Rev Environ Resour</source>
          <year>2014</year>
          <month>10</month>
          <day>17</day>
          <volume>39</volume>
          <issue>1</issue>
          <fpage>641</fpage>
          <lpage>65</lpage>
          <pub-id pub-id-type="doi">10.1146/annurev-environ-030713-154609</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref53">
        <label>53</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Pryor</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Carpenter</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Townsend</surname>
              <given-names>M</given-names>
            </name>
          </person-group>
          <article-title>Outdoor education and bush adventure therapy: a socio-ecological approach to health and wellbeing</article-title>
          <source>Aust J Outdoor Environ Educ</source>
          <year>2005</year>
          <month>04</month>
          <day>1</day>
          <volume>9</volume>
          <issue>1</issue>
          <fpage>3</fpage>
          <lpage>13</lpage>
          <pub-id pub-id-type="doi">10.1007/BF03400807</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref54">
        <label>54</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Maller</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Townsend</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Pryor</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Brown</surname>
              <given-names>P</given-names>
            </name>
            <name name-style="western">
              <surname>St Leger</surname>
              <given-names>L</given-names>
            </name>
          </person-group>
          <article-title>Healthy nature healthy people: 'contact with nature' as an upstream health promotion intervention for populations</article-title>
          <source>Health Promot Int</source>
          <year>2006</year>
          <month>03</month>
          <volume>21</volume>
          <issue>1</issue>
          <fpage>45</fpage>
          <lpage>54</lpage>
          <pub-id pub-id-type="doi">10.1093/heapro/dai032</pub-id>
          <pub-id pub-id-type="medline">16373379</pub-id>
          <pub-id pub-id-type="pii">dai032</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref55">
        <label>55</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Robinson</surname>
              <given-names>T</given-names>
            </name>
          </person-group>
          <article-title>Applying the socio-ecological model to improving fruit and vegetable intake among low-income African Americans</article-title>
          <source>J Community Health</source>
          <year>2008</year>
          <month>07</month>
          <day>2</day>
          <volume>33</volume>
          <issue>6</issue>
          <fpage>395</fpage>
          <lpage>406</lpage>
          <pub-id pub-id-type="doi">10.1007/s10900-008-9109-5</pub-id>
          <pub-id pub-id-type="medline">18594953</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref56">
        <label>56</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Fink</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Damoulas</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Dave</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <article-title>Adaptive Spatio-Temporal Exploratory Models: Hemisphere-Wide Species Distributions From Massively Crowdsourced eBird Data</article-title>
          <source>Proceedings of the Twenty-seventh AAAI Conference on Artificial Intelligence</source>
          <year>2013</year>
          <conf-name>AAAI'13</conf-name>
          <conf-date>July 14–18, 2013</conf-date>
          <conf-loc>Bellevue, Washington, USA</conf-loc>
          <fpage>1284</fpage>
          <lpage>90</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://pdfs.semanticscholar.org/f885/6ff340a3e029041377bfe5db0db73659d8d6.pdf"/>
          </comment>
        </nlm-citation>
      </ref>
      <ref id="ref57">
        <label>57</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Fink</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Hochachka</surname>
              <given-names>WM</given-names>
            </name>
            <name name-style="western">
              <surname>Zuckerberg</surname>
              <given-names>B</given-names>
            </name>
            <name name-style="western">
              <surname>Winkler</surname>
              <given-names>DW</given-names>
            </name>
            <name name-style="western">
              <surname>Shaby</surname>
              <given-names>B</given-names>
            </name>
            <name name-style="western">
              <surname>Munson</surname>
              <given-names>MA</given-names>
            </name>
            <name name-style="western">
              <surname>Hooker</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Riedewald</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Sheldon</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Kelling</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <article-title>Spatiotemporal exploratory models for broad-scale survey data</article-title>
          <source>Ecol Appl</source>
          <year>2010</year>
          <month>12</month>
          <volume>20</volume>
          <issue>8</issue>
          <fpage>2131</fpage>
          <lpage>47</lpage>
          <pub-id pub-id-type="doi">10.1890/09-1340.1</pub-id>
          <pub-id pub-id-type="medline">21265447</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref58">
        <label>58</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hochachka</surname>
              <given-names>WM</given-names>
            </name>
            <name name-style="western">
              <surname>Fink</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Hutchinson</surname>
              <given-names>RA</given-names>
            </name>
            <name name-style="western">
              <surname>Sheldon</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Wong</surname>
              <given-names>WK</given-names>
            </name>
            <name name-style="western">
              <surname>Kelling</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <article-title>Data-intensive science applied to broad-scale citizen science</article-title>
          <source>Trends Ecol Evol</source>
          <year>2012</year>
          <month>02</month>
          <volume>27</volume>
          <issue>2</issue>
          <fpage>130</fpage>
          <lpage>7</lpage>
          <pub-id pub-id-type="doi">10.1016/j.tree.2011.11.006</pub-id>
          <pub-id pub-id-type="medline">22192976</pub-id>
          <pub-id pub-id-type="pii">S0169-5347(11)00329-6</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref59">
        <label>59</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Xue</surname>
              <given-names>Y</given-names>
            </name>
            <name name-style="western">
              <surname>Dilkina</surname>
              <given-names>B</given-names>
            </name>
            <name name-style="western">
              <surname>Damoulas</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Fink</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Gomes</surname>
              <given-names>CP</given-names>
            </name>
            <name name-style="western">
              <surname>Kelling</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <article-title>Improving Your Chances: Boosting Citizen Science Discovery</article-title>
          <source>Proceedings of the 1st AAAI Conference on Human Computation and Crowdsourcing</source>
          <year>2013</year>
          <conf-name>HCOMP'13</conf-name>
          <conf-date>November 7-9, 2013</conf-date>
          <conf-loc>Palm Springs, California, USA</conf-loc>
          <fpage>198</fpage>
          <lpage>206</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.cs.cornell.edu/~yexiang/publications/hcomp13_birdassist/xue.pdf"/>
          </comment>
        </nlm-citation>
      </ref>
      <ref id="ref60">
        <label>60</label>
        <nlm-citation citation-type="book">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hufford</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Taylor</surname>
              <given-names>B</given-names>
            </name>
          </person-group>
          <person-group person-group-type="editor">
            <name name-style="western">
              <surname>Craynon</surname>
              <given-names>JR</given-names>
            </name>
          </person-group>
          <article-title>Edgework in boundary crossings: assessing foundations for public ecology in the Appalachian region</article-title>
          <source>Environmental Considerations in Energy Production</source>
          <year>2013</year>
          <publisher-loc>Washington, DC</publisher-loc>
          <publisher-name>Mining Development Services</publisher-name>
          <fpage>99</fpage>
          <lpage>110</lpage>
        </nlm-citation>
      </ref>
      <ref id="ref61">
        <label>61</label>
        <nlm-citation citation-type="book">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Tidball</surname>
              <given-names>KG</given-names>
            </name>
            <name name-style="western">
              <surname>Krasny</surname>
              <given-names>ME</given-names>
            </name>
          </person-group>
          <person-group person-group-type="editor">
            <name name-style="western">
              <surname>Dickinson</surname>
              <given-names>JL</given-names>
            </name>
            <name name-style="western">
              <surname>Bonney</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Fitzpatrick</surname>
              <given-names>JW</given-names>
            </name>
          </person-group>
          <article-title>What role for citizen science in disaster conflict</article-title>
          <source>Citizen Science: Public Participation in Environmental Research</source>
          <year>2012</year>
          <publisher-loc>Ithaca, New York</publisher-loc>
          <publisher-name>Cornell University Press</publisher-name>
          <fpage>226</fpage>
          <lpage>34</lpage>
        </nlm-citation>
      </ref>
      <ref id="ref62">
        <label>62</label>
        <nlm-citation citation-type="book">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Osgood</surname>
              <given-names>ND</given-names>
            </name>
          </person-group>
          <person-group person-group-type="editor">
            <name name-style="western">
              <surname>El-Sayed</surname>
              <given-names>AM</given-names>
            </name>
            <name name-style="western">
              <surname>Galea</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <article-title>Frontiers in health modeling</article-title>
          <source>Systems Science and Population Health</source>
          <year>2017</year>
          <publisher-loc>Oxford, United Kingdom</publisher-loc>
          <publisher-name>Oxford University Press</publisher-name>
          <fpage>191</fpage>
          <lpage>216</lpage>
        </nlm-citation>
      </ref>
      <ref id="ref63">
        <label>63</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Wald</surname>
              <given-names>DM</given-names>
            </name>
            <name name-style="western">
              <surname>Longo</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Dobell</surname>
              <given-names>AR</given-names>
            </name>
          </person-group>
          <article-title>Design principles for engaging and retaining virtual citizen scientists</article-title>
          <source>Conserv Biol</source>
          <year>2016</year>
          <month>06</month>
          <volume>30</volume>
          <issue>3</issue>
          <fpage>562</fpage>
          <lpage>70</lpage>
          <pub-id pub-id-type="doi">10.1111/cobi.12627</pub-id>
          <pub-id pub-id-type="medline">26365126</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref64">
        <label>64</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Lunden</surname>
              <given-names>I</given-names>
            </name>
          </person-group>
          <source>Techcrunch</source>
          <year>2015</year>
          <access-date>2019-02-24</access-date>
          <comment>6.1B Smartphone Users Globally By 2020, Overtaking Basic Fixed Phone Subscriptions 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://techcrunch.com/2015/06/02/6-1b-smartphone-users-globally-by-2020-overtaking-basic-fixed-phone-subscriptions/">http://techcrunch.com/2015/06/02/6-1b-smartphone-users-globally-by-2020-overtaking-basic-fixed-phone-subscriptions/</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref65">
        <label>65</label>
        <nlm-citation citation-type="web">
          <source>Catalyst: Digital Marketing Services to Fuel Results</source>
          <year>2015</year>
          <access-date>2019-02-24</access-date>
          <comment>With Growth Comes Change: The Evolving Mobile Landscape in 2015 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://catalyst.ca/2015-canadian-smartphone-market/">http://catalyst.ca/2015-canadian-smartphone-market/</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref66">
        <label>66</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Raghupathi</surname>
              <given-names>W</given-names>
            </name>
            <name name-style="western">
              <surname>Raghupathi</surname>
              <given-names>V</given-names>
            </name>
          </person-group>
          <article-title>Big data analytics in healthcare: promise and potential</article-title>
          <source>Health Inf Sci Syst</source>
          <year>2014</year>
          <volume>2</volume>
          <fpage>3</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/25825667"/>
          </comment>
          <pub-id pub-id-type="doi">10.1186/2047-2501-2-3</pub-id>
          <pub-id pub-id-type="medline">25825667</pub-id>
          <pub-id pub-id-type="pii">14</pub-id>
          <pub-id pub-id-type="pmcid">PMC4341817</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref67">
        <label>67</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Katapally</surname>
              <given-names>T</given-names>
            </name>
          </person-group>
          <source>SMART: A Mobile Health and Citizen Science Platform</source>
          <year>2019</year>
          <access-date>2019-07-12</access-date>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.smartstudysask.com">https://www.smartstudysask.com</ext-link>
          </comment>
        </nlm-citation>
      </ref>
      <ref id="ref68">
        <label>68</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Manjoo</surname>
              <given-names>F</given-names>
            </name>
          </person-group>
          <source>The New York Times</source>
          <year>2017</year>
          <access-date>2019-03-09</access-date>
          <comment>Tech’s Frightful Five: They’ve Got Us 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.nytimes.com/2017/05/10/technology/techs-frightful-five-theyve-got-us.html">https://www.nytimes.com/2017/05/10/technology/techs-frightful-five-theyve-got-us.html</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref69">
        <label>69</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Katapally</surname>
              <given-names>T</given-names>
            </name>
          </person-group>
          <source>SMART: A Mobile Health and Citizen Science Platform</source>
          <year>2019</year>
          <access-date>2019-07-12</access-date>
          <comment>The SMART Platform 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.smartstudysask.com/about">https://www.smartstudysask.com/about</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref70">
        <label>70</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Katapally</surname>
              <given-names>T</given-names>
            </name>
          </person-group>
          <source>SMART: A Mobile Health and Citizen Science Platform</source>
          <year>2019</year>
          <access-date>2019-07-12</access-date>
          <comment>SMART Indigenous Youth 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.smartstudysask.com/smart-first-nations-youth">https://www.smartstudysask.com/smart-first-nations-youth</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref71">
        <label>71</label>
        <nlm-citation citation-type="web">
          <source>Métis Nation of Ontario</source>
          <year>2013</year>
          <access-date>2019-07-04</access-date>
          <comment>Canada’s Most Vulnerable: Improving Health Care for First Nations, Inuit, and Métis Seniors 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://www.metisnation.org/media/422632/senior_ab_report_2013_en_final.pdf">http://www.metisnation.org/media/422632/senior_ab_report_2013_en_final.pdf</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref72">
        <label>72</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>O'Donnell</surname>
              <given-names>V</given-names>
            </name>
            <name name-style="western">
              <surname>Wallace</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <source>Statistics Canada</source>
          <year>2011</year>
          <access-date>2019-07-04</access-date>
          <comment>First Nations, Métis and Inuit Women 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www150.statcan.gc.ca/n1/pub/89-503-x/2010001/article/11442-eng.htm">https://www150.statcan.gc.ca/n1/pub/89-503-x/2010001/article/11442-eng.htm</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref73">
        <label>73</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Katapally</surname>
              <given-names>T</given-names>
            </name>
          </person-group>
          <source>SMART: A Mobile Health and Citizen Science Platform</source>
          <year>2019</year>
          <access-date>2019-07-12</access-date>
          <comment>SMART Adults 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.smartstudysask.com/copy-of-smart-youth">https://www.smartstudysask.com/copy-of-smart-youth</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref74">
        <label>74</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Katapally</surname>
              <given-names>T</given-names>
            </name>
          </person-group>
          <source>SMART: A Mobile Health and Citizen Science Platform</source>
          <year>2019</year>
          <access-date>2019-07-12</access-date>
          <comment>SMART Youth 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.smartstudysask.com/smart-youth">https://www.smartstudysask.com/smart-youth</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref75">
        <label>75</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Katapally</surname>
              <given-names>T</given-names>
            </name>
          </person-group>
          <source>SMART: A Mobile Health and Citizen Science Platform</source>
          <year>2019</year>
          <access-date>2019-07-12</access-date>
          <comment>SMART India 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.smartstudysask.com/smart-india">https://www.smartstudysask.com/smart-india</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref76">
        <label>76</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Katapally</surname>
              <given-names>T</given-names>
            </name>
          </person-group>
          <source>SMART: A Mobile Health and Citizen Science Platform</source>
          <year>2019</year>
          <access-date>2019-07-12</access-date>
          <comment>SMART Response 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.smartstudysask.com/copy-of-smart-adults">https://www.smartstudysask.com/copy-of-smart-adults</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref77">
        <label>77</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Katapally</surname>
              <given-names>TR</given-names>
            </name>
            <name name-style="western">
              <surname>Chu</surname>
              <given-names>LM</given-names>
            </name>
          </person-group>
          <article-title>Methodology to derive objective screen-state from smartphones: a SMART platform study</article-title>
          <source>Int J Environ Res Public Health</source>
          <year>2019</year>
          <month>06</month>
          <day>27</day>
          <volume>16</volume>
          <issue>13</issue>
          <fpage>E2275</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://www.mdpi.com/resolver?pii=ijerph16132275"/>
          </comment>
          <pub-id pub-id-type="doi">10.3390/ijerph16132275</pub-id>
          <pub-id pub-id-type="medline">31252617</pub-id>
          <pub-id pub-id-type="pii">ijerph16132275</pub-id>
          <pub-id pub-id-type="pmcid">PMC6651165</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref78">
        <label>78</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Martinez-Moyano</surname>
              <given-names>IJ</given-names>
            </name>
          </person-group>
          <article-title>Documentation for model transparency</article-title>
          <source>Syst Dyn Rev</source>
          <year>2012</year>
          <month>05</month>
          <day>22</day>
          <volume>28</volume>
          <issue>2</issue>
          <fpage>199</fpage>
          <lpage>208</lpage>
          <pub-id pub-id-type="doi">10.1002/sdr.1471</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref79">
        <label>79</label>
        <nlm-citation citation-type="book">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Cooper</surname>
              <given-names>C</given-names>
            </name>
          </person-group>
          <source>Citizen Science: How Ordinary People are Changing the Face of Discovery</source>
          <year>2016</year>
          <publisher-loc>New York</publisher-loc>
          <publisher-name>The Overlook Press</publisher-name>
        </nlm-citation>
      </ref>
      <ref id="ref80">
        <label>80</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Scott</surname>
              <given-names>WR</given-names>
            </name>
          </person-group>
          <article-title>Approaching adulthood: the maturing of institutional theory</article-title>
          <source>Theor Soc</source>
          <year>2008</year>
          <month>06</month>
          <day>11</day>
          <volume>37</volume>
          <issue>5</issue>
          <fpage>427</fpage>
          <lpage>42</lpage>
          <pub-id pub-id-type="doi">10.1007/s11186-008-9067-z</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref81">
        <label>81</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Weick</surname>
              <given-names>KE</given-names>
            </name>
            <name name-style="western">
              <surname>Quinn</surname>
              <given-names>RE</given-names>
            </name>
          </person-group>
          <article-title>Organizational change and development</article-title>
          <source>Annu Rev Psychol</source>
          <year>1999</year>
          <volume>50</volume>
          <fpage>361</fpage>
          <lpage>86</lpage>
          <pub-id pub-id-type="doi">10.1146/annurev.psych.50.1.361</pub-id>
          <pub-id pub-id-type="medline">15012461</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref82">
        <label>82</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Freitag</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Pfeffer</surname>
              <given-names>MJ</given-names>
            </name>
          </person-group>
          <article-title>Process, not product: investigating recommendations for improving citizen science 'success'</article-title>
          <source>PLoS One</source>
          <year>2013</year>
          <volume>8</volume>
          <issue>5</issue>
          <fpage>e64079</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://dx.plos.org/10.1371/journal.pone.0064079"/>
          </comment>
          <pub-id pub-id-type="doi">10.1371/journal.pone.0064079</pub-id>
          <pub-id pub-id-type="medline">23691154</pub-id>
          <pub-id pub-id-type="pii">PONE-D-13-04083</pub-id>
          <pub-id pub-id-type="pmcid">PMC3655009</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref83">
        <label>83</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Fuller</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Hermeston</surname>
              <given-names>W</given-names>
            </name>
            <name name-style="western">
              <surname>Passey</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Fallon</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Muyambi</surname>
              <given-names>K</given-names>
            </name>
          </person-group>
          <article-title>Acceptability of participatory social network analysis for problem-solving in Australian Aboriginal health service partnerships</article-title>
          <source>BMC Health Serv Res</source>
          <year>2012</year>
          <month>06</month>
          <day>10</day>
          <volume>12</volume>
          <fpage>152</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-12-152"/>
          </comment>
          <pub-id pub-id-type="doi">10.1186/1472-6963-12-152</pub-id>
          <pub-id pub-id-type="medline">22682504</pub-id>
          <pub-id pub-id-type="pii">1472-6963-12-152</pub-id>
          <pub-id pub-id-type="pmcid">PMC3472193</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref84">
        <label>84</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Alabri</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Hunter</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <article-title>Enhancing the Quality and Trust of Citizen Science Data</article-title>
          <source>Proeedings of the Sixth International Conference on e-Science</source>
          <year>2011</year>
          <month>01</month>
          <day>20</day>
          <conf-name>eScience'11</conf-name>
          <conf-date>December 7-10, 2010</conf-date>
          <conf-loc>Brisbane, QLD, Australia</conf-loc>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://ieeexplore.ieee.org/document/5693902"/>
          </comment>
          <pub-id pub-id-type="doi">10.1109/eScience.2010.33</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref85">
        <label>85</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hunter</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Alabri</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>van Ingen</surname>
              <given-names>C</given-names>
            </name>
          </person-group>
          <article-title>Assessing the quality and trustworthiness of citizen science data</article-title>
          <source>Concurr Comp-Pract E</source>
          <year>2012</year>
          <month>09</month>
          <day>13</day>
          <volume>25</volume>
          <issue>4</issue>
          <fpage>454</fpage>
          <lpage>66</lpage>
          <pub-id pub-id-type="doi">10.1002/cpe.2923</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref86">
        <label>86</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Tulloch</surname>
              <given-names>AI</given-names>
            </name>
            <name name-style="western">
              <surname>Possingham</surname>
              <given-names>HP</given-names>
            </name>
            <name name-style="western">
              <surname>Joseph</surname>
              <given-names>LN</given-names>
            </name>
            <name name-style="western">
              <surname>Szabo</surname>
              <given-names>JK</given-names>
            </name>
            <name name-style="western">
              <surname>Martin</surname>
              <given-names>TG</given-names>
            </name>
          </person-group>
          <article-title>Realising the full potential of citizen science monitoring programs</article-title>
          <source>Biol Conserv</source>
          <year>2013</year>
          <month>09</month>
          <volume>165</volume>
          <fpage>128</fpage>
          <lpage>38</lpage>
          <pub-id pub-id-type="doi">10.1016/j.biocon.2013.05.025</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref87">
        <label>87</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Bonter</surname>
              <given-names>DN</given-names>
            </name>
            <name name-style="western">
              <surname>Cooper</surname>
              <given-names>CB</given-names>
            </name>
          </person-group>
          <article-title>Data validation in citizen science: a case study from project FeederWatch</article-title>
          <source>Front Ecol Environ</source>
          <year>2012</year>
          <month>08</month>
          <volume>10</volume>
          <issue>6</issue>
          <fpage>305</fpage>
          <lpage>7</lpage>
          <pub-id pub-id-type="doi">10.1890/110273</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref88">
        <label>88</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hampton</surname>
              <given-names>SE</given-names>
            </name>
            <name name-style="western">
              <surname>Strasser</surname>
              <given-names>CA</given-names>
            </name>
            <name name-style="western">
              <surname>Tewksbury</surname>
              <given-names>JJ</given-names>
            </name>
            <name name-style="western">
              <surname>Gram</surname>
              <given-names>WK</given-names>
            </name>
            <name name-style="western">
              <surname>Budden</surname>
              <given-names>AE</given-names>
            </name>
            <name name-style="western">
              <surname>Batcheller</surname>
              <given-names>AL</given-names>
            </name>
            <name name-style="western">
              <surname>Duke</surname>
              <given-names>CS</given-names>
            </name>
            <name name-style="western">
              <surname>Porter</surname>
              <given-names>JH</given-names>
            </name>
          </person-group>
          <article-title>Big data and the future of ecology</article-title>
          <source>Front Ecol Environ</source>
          <year>2013</year>
          <month>04</month>
          <volume>11</volume>
          <issue>3</issue>
          <fpage>156</fpage>
          <lpage>62</lpage>
          <pub-id pub-id-type="doi">10.1890/120103</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref89">
        <label>89</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Szabo</surname>
              <given-names>JK</given-names>
            </name>
            <name name-style="western">
              <surname>Vesk</surname>
              <given-names>PA</given-names>
            </name>
            <name name-style="western">
              <surname>Baxter</surname>
              <given-names>PW</given-names>
            </name>
            <name name-style="western">
              <surname>Possingham</surname>
              <given-names>HP</given-names>
            </name>
          </person-group>
          <article-title>Regional avian species declines estimated from volunteer-collected long-term data using list length analysis</article-title>
          <source>Ecol Appl</source>
          <year>2010</year>
          <month>12</month>
          <volume>20</volume>
          <issue>8</issue>
          <fpage>2157</fpage>
          <lpage>69</lpage>
          <pub-id pub-id-type="doi">10.1890/09-0877.1</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref90">
        <label>90</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Worthington</surname>
              <given-names>JP</given-names>
            </name>
            <name name-style="western">
              <surname>Silvertown</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Cook</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Cameron</surname>
              <given-names>RA</given-names>
            </name>
            <name name-style="western">
              <surname>Dodd</surname>
              <given-names>ME</given-names>
            </name>
            <name name-style="western">
              <surname>Greenwood</surname>
              <given-names>RM</given-names>
            </name>
            <name name-style="western">
              <surname>Mcconway</surname>
              <given-names>KJ</given-names>
            </name>
            <name name-style="western">
              <surname>Skelton</surname>
              <given-names>PW</given-names>
            </name>
          </person-group>
          <article-title>Evolution MegaLab: a case study in citizen science methods</article-title>
          <source>Methods Ecol Evol</source>
          <year>2012</year>
          <volume>3</volume>
          <issue>2</issue>
          <fpage>3030</fpage>
          <lpage>9</lpage>
          <pub-id pub-id-type="doi">10.1111/j.2041-210X.2011.00164.x</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref91">
        <label>91</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Wiggins</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Bonney</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Graham</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>Henderson</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Kelling</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>LeBuhn</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Littauer</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Lotts</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Michener</surname>
              <given-names>W</given-names>
            </name>
            <name name-style="western">
              <surname>Newman</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Russell</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>Stevenson</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Weltzin</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <source>DataONE</source>
          <year>2013</year>
          <access-date>2019-07-12</access-date>
          <comment>Data Management Guide for Public Participation in Scientific Research 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.dataone.org/sites/all/documents/DataONE-PPSR-DataManagementGuide.pdf">https://www.dataone.org/sites/all/documents/DataONE-PPSR-DataManagementGuide.pdf</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref92">
        <label>92</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Büchi</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Just</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>Latzer</surname>
              <given-names>M</given-names>
            </name>
          </person-group>
          <article-title>Modeling the second-level digital divide: A five-country study of social differences in Internet use</article-title>
          <source>New Media &amp; Society</source>
          <year>2016</year>
          <month>07</month>
          <day>10</day>
          <volume>18</volume>
          <issue>11</issue>
          <fpage>2703</fpage>
          <lpage>22</lpage>
          <pub-id pub-id-type="doi">10.1177/1461444815604154</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref93">
        <label>93</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Tsetsi</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>Rains</surname>
              <given-names>SA</given-names>
            </name>
          </person-group>
          <article-title>Smartphone Internet access and use: Extending the digital divide and usage gap</article-title>
          <source>Mobile Media &amp; Communication</source>
          <year>2017</year>
          <month>06</month>
          <day>13</day>
          <volume>5</volume>
          <issue>3</issue>
          <fpage>239</fpage>
          <lpage>55</lpage>
          <pub-id pub-id-type="doi">10.1177/2050157917708329</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref94">
        <label>94</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hilbert</surname>
              <given-names>M</given-names>
            </name>
          </person-group>
          <article-title>The bad news is that the digital access divide is here to stay: Domestically installed bandwidths among 172 countries for 1986–2014</article-title>
          <source>Telecommunications Policy</source>
          <year>2016</year>
          <month>06</month>
          <volume>40</volume>
          <issue>6</issue>
          <fpage>567</fpage>
          <lpage>81</lpage>
          <pub-id pub-id-type="doi">10.1016/j.telpol.2016.01.006</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref95">
        <label>95</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Galloway</surname>
              <given-names>L</given-names>
            </name>
          </person-group>
          <source>BBC</source>
          <year>2019</year>
          <access-date>2019-07-07</access-date>
          <comment>Five Countries on the Frontline of Tech 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://www.bbc.com/travel/story/20190630-five-countries-on-the-frontline-of-tech">http://www.bbc.com/travel/story/20190630-five-countries-on-the-frontline-of-tech</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref96">
        <label>96</label>
        <nlm-citation citation-type="web">
          <source>Office of the United Nations High Commissioner for Human Rights</source>
          <year>2011</year>
          <access-date>2019-07-12</access-date>
          <comment>Report of the Special Rapporteur on the Promotion and Protection of the Right to Freedom Of Opinion and Expression, Frank La Rue 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www2.ohchr.org/english/bodies/hrcouncil/docs/17session/A.HRC.17.27_en.pdf">https://www2.ohchr.org/english/bodies/hrcouncil/docs/17session/A.HRC.17.27_en.pdf</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref97">
        <label>97</label>
        <nlm-citation citation-type="web">
          <source>BBC</source>
          <year>2010</year>
          <access-date>2019-03-08</access-date>
          <comment>Internet Access is 'a Fundamental Right' 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://news.bbc.co.uk/2/hi/technology/8548190.stm">http://news.bbc.co.uk/2/hi/technology/8548190.stm</ext-link></comment>
        </nlm-citation>
      </ref>
      <ref id="ref98">
        <label>98</label>
        <nlm-citation citation-type="web">
          <source>Budget 2019 - Canada</source>
          <year>2019</year>
          <access-date>2019-07-12</access-date>
          <comment>Building a Better Canada: Universal High-Speed Internet 
        <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.budget.gc.ca/2019/docs/nrc/infrastructure-infrastructures-internet-en.html">https://www.budget.gc.ca/2019/docs/nrc/infrastructure-infrastructures-internet-en.html</ext-link></comment>
        </nlm-citation>
      </ref>
    </ref-list>
  </back>
</article>
