Document Type
Article
Abstract
In thematic analysis, themes construction can be performed manually by the researcher or automatically by a computer. Both methods have strengths and weaknesses. This article introduces a strategy that involves the role of both researcher and computer to construct themes from qualitative data in a rapid, transparent, and rigorous manner. The strategy uses network analysis and is demonstrated by employing a case study on students' perceptions of online distance learning they experienced during the COVID-19 pandemic. The themes-construction strategy consists of four systematic phases, namely (1) determining unit of analysis and coding; (2) constructing the code co-occurrence matrix; (3) conducting network analysis; and (4) generating, reviewing, and reporting the themes. The strategy is successfully demonstrated in generating themes from the data with modularity value Q = 0.34. The application of network analysis in this strategy allows researchers to automatically generate themes from qualitative data using mathematical algorithms, represent these themes visually using network graph, and interpret the themes to answer the research questions.
First Page
177
Last Page
189
Issue
2
Volume
24
Digital Object Identifier (DOI)
10.21831/pep.v24i2.33912
Recommended Citation
Kristanto, Yosep Dwi and Padmi, Russasmita Sri
(2020)
"Using network analysis for rapid, transparent, and rigorous thematic analysis: A case study of online distance learning,"
Jurnal Penelitian dan Evaluasi Pendidikan: Vol. 24:
Iss.
2, Article 6.
DOI: 10.21831/pep.v24i2.33912
Available at:
https://scholarhub.uny.ac.id/jpep/vol24/iss2/6
References
Aguinis, H., & Solarino, A. M. (2019). Transparency and replicability in qualitative research: The case of interviews with elite informants. Strategic Management Journal, 40, 1291-1315. https://doi.org/10.1002/smj.3015
Attride-Stirling, J. (2001). Thematic networks: An analytic tool for qualitative research. Qualitative Research, 1(3), 385-405. https://doi.org/10.1177/146879410100100307
Bastian, M., Heymann, S., & Jacomy, M. (2009). Gephi: An open source software for exploring and manipulating networks. International AAAI Conference on Weblogs and Social Media. http://www.aaai.org/ocs/index.php/ICWSM/09/paper/view/154
Blondel, V. D., Guillaume, J.-L., Lambiotte, R., & Lefebvre, E. (2008). Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 2008(10), P10008. https://doi.org/10.1088/1742-5468/2008/10/P10008
Brandes, U., Delling, D., Gaertler, M., Gorke, R., Hoefer, M., Nikoloski, Z., & Wagner, D. (2008). On modularity clustering. IEEE Transactions on Knowledge and Data Engineering, 20(2), 172-188. https://doi.org/10.1109/TKDE.2007.190689
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
Braun, V., & Clarke, V. (2012). Thematic analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. (pp. 57-71). American Psychological Association. https://doi.org/10.1037/13620-004
Brennan, L., Cusack, T., Delahunt, E., Kuznesof, S., & Donnelly, S. (2019). Academics' conceptualisations of the research-teaching nexus in a research-intensive Irish university: A dynamic framework for growth & development. Learning and Instruction, 60, 301-309. https://doi.org/10.1016/j.learninstruc.2017.10.005
Bringer, J. D., Johnston, L. H., & Brackenridge, C. H. (2006). Using computer-assisted qualitative data analysis software to develop a grounded theory project. Field Methods, 18(3), 245-266. https://doi.org/10.1177/1525822X06287602
Burgess-Allen, J., & Owen-Smith, V. (2010). Using mind mapping techniques for rapid qualitative data analysis in public participation processes. Health Expectations, 13(4), 406-415. https://doi.org/10.1111/j.1369-7625.2010.00594.x
Burns, S., & Schubotz, D. (2009). Demonstrating the merits of the peer research process: A Northern Ireland case study. Field Methods, 21(3), 309-326. https://doi.org/10.1177/1525822X09333514
Chandra, Y., & Shang, L. (2019). An overview of R and RQDA: An open-source CAQDAS platform. In Qualitative research using R: A systematic approach (pp. 47-51). Springer Singapore. https://doi.org/10.1007/978-981-13-3170-1_4
Chen, K., Zhang, Y., & Fu, X. (2019). International research collaboration: An emerging domain of innovation studies? Research Policy, 48(1), 149-168. https://doi.org/10.1016/j.respol.2018.08.005
Clarke, V., & Braun, V. (2013). Teaching thematic analysis: Overcoming challenges and developing strategies for effective learning. The Psychologist, 26(2), 120-123. https://uwe-repository.worktribe.com/output/937596/teaching-thematic-analysis-overcoming-challenges-and-developing-strategies-for-effective-learning
Drijvers, P., Grauwin, S., & Trouche, L. (2020). When bibliometrics met mathematics education research: the case of instrumental orchestration. ZDM. https://doi.org/10.1007/s11858-020-01169-3
Farrell, O., & Seery, A. (2019). "I am not simply learning and regurgitating information, I am also learning about myself": learning portfolio practice and online distance students. Distance Education, 40(1), 76-97. https://doi.org/10.1080/01587919.2018.1553565
Fielding, N. G., & Lee, R. M. (2002). New patterns in the adoption and use of qualitative software. Field Methods, 14(2), 197-216. https://doi.org/10.1177/1525822X02014002005
Frandsen, T. (2017). Evolution of modularity literature: A 25-year bibliometric analysis. International Journal of Operations & Production Management, 37(6), 703-747. https://doi.org/10.1108/IJOPM-06-2015-0366
Galdas, P. (2017). Revisiting bias in qualitative research. International Journal of Qualitative Methods, 16(1), 160940691774899. https://doi.org/10.1177/1609406917748992
Good, B. H., de Montjoye, Y.-A., & Clauset, A. (2010). Performance of modularity maximization in practical contexts. Physical Review E, 81(4), 046106. https://doi.org/10.1103/PhysRevE.81.046106
Hannes, K., & Macaitis, K. (2012). A move to more systematic and transparent approaches in qualitative evidence synthesis: Update on a review of published papers. Qualitative Research, 12(4), 402-442. https://doi.org/10.1177/1468794111432992
Harary, F. (2018). Graph theory. CRC Press.
Haskins Lisle, A., Merenda, C., & Gabbard, J. (2020). Using affinity diagramming to generate a codebook: a case study on young military veterans and community reintegration. Qualitative Research, 20(4), 396-413. https://doi.org/10.1177/1468794119851324
Holloway, I., & Todres, L. (2003). The status of method: Flexibility, consistency and coherence. Qualitative Research, 3(3), 345-357. https://doi.org/10.1177/1468794103033004
Jackson, K., & Bazeley, P. (2019). Qualitative data analysis with NVivo. SAGE.
Jackson, K. M., & Trochim, W. M. K. (2002). Concept mapping as an alternative approach for the analysis of open-ended survey responses. Organizational Research Methods, 5(4), 307-336. https://doi.org/10.1177/109442802237114
Jamieson, S. (2016). Analyse qualitative data. Education for Primary Care, 27(5), 398-402. https://doi.org/10.1080/14739879.2016.1217430
John, W. S., & Johnson, P. (2000). The pros and cons of data analysis software for qualitative research. Journal of Nursing Scholarship, 32(4), 393-397. https://doi.org/10.1111/j.1547-5069.2000.00393.x
Kessler, M. M. (1963). Bibliographic coupling between scientific papers. American Documentation, 14(1), 10-25. https://doi.org/10.1002/asi.5090140103
Kiger, M. E., & Varpio, L. (2020). Thematic analysis of qualitative data: AMEE Guide No. 131. Medical Teacher, 1-9. https://doi.org/10.1080/0142159X.2020.1755030
King, A. (2010). 'Membership matters': Applying membership categorisation analysis (MCA) to qualitative data using computer"assisted qualitative data analysis (CAQDAS) software. International Journal of Social Research Methodology, 13(1), 1-16. https://doi.org/10.1080/13645570802576575
King, N., & Brooks, J. (2018). Thematic analysis in organisational research. In C. Cassell, A. L. Cunliffe, & G. Grandy (Eds.), The SAGE handbook of qualitative business and management research methods: Methods and challenges (pp. 219-236). SAGE Publications Ltd. https://doi.org/10.4135/9781526430236.n14
Kristanto, Y. D., & Santoso, E. B. (2020). Towards a mathematics textbook for supporting 21st century learning: The student perspective. Journal of Physics: Conference Series, 1657, 012037. https://doi.org/10.1088/1742-6596/1657/1/012037
Kristanto, Y. D. (2018). Technology-enhanced pre-instructional peer assessment: Exploring students' perceptions in a Statistical Methods course. REiD (Research and Evaluation in Education), 4(2), 105-116. https://doi.org/10.21831/reid.v4i2.20951
Kristanto, Y. D., Utomo, B., & Sulistyani, N. (2020). Towards instructional design principles for online distance learning. https://doi.org/10.17605/OSF.IO/S2CN8
LeCompte, M. D., & Schensul, J. J. (1999). Analyzing & interpreting ethnographic data. AltaMira Press.
Lee, V., Coombe, L., Mahoney, R., Allen, C., & Robinson, P. (2018). Incorporating indigenous and non-indigenous worldviews through innovative text analysis: An evaluation of indigenous public health curricula. International Journal of Qualitative Methods, 17(1), 160940691881937. https://doi.org/10.1177/1609406918819377
Littenberg-Tobias, J., & Reich, J. (2020). Evaluating access, quality, and equity in online learning: A case study of a MOOC-based blended professional degree program. The Internet and Higher Education, 47, 100759. https://doi.org/10.1016/j.iheduc.2020.100759
Lund, K., Chen, B., & Grauwin, S. (2018). The potential of interdisciplinary in MOOC research. Proceedings of the Fifth Annual ACM Conference on Learning at Scale, 1-10. https://doi.org/10.1145/3231644.3231661
McCain, K. W. (1990). Mapping authors in intellectual space: A technical overview. Journal of the American Society for Information Science, 41(6), 433-443. https://doi.org/10.1002/(SICI)1097-4571(199009)41:6<433::AID-ASI11>3.0.CO;2-Q
Newman, M. E. J. (2004). Analysis of weighted networks. Physical Review E, 70(5), 056131. https://doi.org/10.1103/PhysRevE.70.056131
Newman, M. E. J. (2006). Modularity and community structure in networks. Proceedings of the National Academy of Sciences, 103(23), 8577-8582. https://doi.org/10.1073/pnas.0601602103
Newman, M. E. J. (2010). Networks: An introduction. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199206650.001.0001
Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1), 1-13. https://doi.org/10.1177/1609406917733847
Opsahl, T., Agneessens, F., & Skvoretz, J. (2010). Node centrality in weighted networks: Generalizing degree and shortest paths. Social Networks, 32(3), 245-251. https://doi.org/10.1016/j.socnet.2010.03.006
Pokorny, J. J., Norman, A., Zanesco, A. P., Bauer-Wu, S., Sahdra, B. K., & Saron, C. D. (2018). Network analysis for the visualization and analysis of qualitative data. Psychological Methods, 23(1), 169-183. https://doi.org/10.1037/met0000129
Rose, J., & Johnson, C. W. (2020). Contextualizing reliability and validity in qualitative research: toward more rigorous and trustworthy qualitative social science in leisure research. Journal of Leisure Research, 51(4), 432-451. https://doi.org/10.1080/00222216.2020.1722042
Ryan, G. W., & Bernard, H. R. (2003). Techniques to identify themes. Field Methods, 15(1), 85-109. https://doi.org/10.1177/1525822X02239569
Salmona, M., Lieber, E., & Kaczynski, D. (2019). Qualitative and mixed methods data analysis using Dedoose: A practical approach for research across the social sciences. SAGE.
Scott, J., & Carrington, P. (2014). The SAGE handbook of social network analysis. SAGE Publications Ltd. https://doi.org/10.4135/9781446294413
Small, H. (1973). Co-citation in the scientific literature: A new measure of the relationship between two documents. Journal of the American Society for Information Science, 24(4), 265-269. https://doi.org/10.1002/asi.4630240406
Stopar, K., & Bartol, T. (2019). Digital competences, computer skills and information literacy in secondary education: mapping and visualization of trends and concepts. Scientometrics, 118(2), 479-498. https://doi.org/10.1007/s11192-018-2990-5
Thompson, R. (2002). Reporting the results of computer-assisted analysis of qualitative research data. Forum: Qualitative Social Research, 3(2), Art. 25. https://doi.org/10.17169/fqs-3.2.864
Tuval-Mashiach, R. (2017). Raising the curtain: The importance of transparency in qualitative research. Qualitative Psychology, 4(2), 126-138. https://doi.org/10.1037/qup0000062
Watkins, D. C. (2017). Rapid and rigorous qualitative data analysis. International Journal of Qualitative Methods, 16(1), 160940691771213. https://doi.org/10.1177/1609406917712131
Zhao, D., & Logan, E. (2002). Citation analysis using scientific publications on the web as data source: A case study in the XML research area. Scientometrics, 54(3), 449-472. https://doi.org/10.1023/A%3A1016090601710