SATLE 2023 Funded Project:
New: Enhancing Teaching and Learning of Data Literacy and Analytics Module (CA179) Using Co-creation of Formative and Summative Assessments, 2024
As a result of the feedback collected from students in 20203 and 2024, I proposed an intervention to involve students in co-creating formative assessment questions. In December 2023, I received SATLE Funding from DCU for the implementation of the proposal.
In May 2024, I hired three undergraduate students who took CA179 in the 2023/24 academic year. The selection of these students was based on their final results and the level of engagement they demonstrated in completing their module. They are selected from different programs to ensure diversity with two Female and one Male student. I also recruited a student who volunteered to work on this project. The student agreed to work even without compensation which made the group dynamic with three female and two male members (including the module coordinator).
The four students spent a combined 400 hours together in four weeks to co-create assessment pieces for seven of the 14 microcurricula included in CA179. The co-creation process involved three days of training sessions with the module coordinator followed by three weeks of content review, question design, peer review and selection of questions to be included in the module.
Throughout the project, the interns reviewed the content of the seven microcurricula, creating 350 formative assessment pieces evenly distributed across the topics, and producing seven topic summaries.
The 350 Formative assessment questions are implemented on the 2024/25 instance of the module allowing current students to benefit from the intervention. The links to CA179 (now CSC1015) are available here and anyone interested is welcome to enrol on the module. Please, send your request to Yalemisewm.abgaz@dcu.ie
Click here to enrol on Data Literacy and Analytics for the 21st Century module
This initiative brought multiple benefits.
- It significantly enhanced the asynchronous and online learning experience by adding engaging, interactive assessments. It offers students another level resource to prepare themselves for the summative assessment questions and helps them evaluate their learning before taking the final assessment in each microcurricula. Although it requires a formal evaluation, preliminary statistics show that student engagement is higher.
- It allowed me to work closely with students, acquiring valuable insights into their learning experiences and challenges.
- It demonstrated my commitment to inclusivity and equity by giving students the opportunity to co-create resources, aligning with inclusive educational principles.
Additionally, this intervention improved academic integrity by reducing the likelihood of assessment recycling and provided a framework for designing assessments for similar modules.
The students also wrote their reflective journals about their experiences and provided me with useful insights for future co-creation of formative and summative assessments. They pointed out that this gave them a very good experience on how educational resources are produced and gained essential skills in using educational technologies to enhance the experience of students.
The project was completed and implemented on August 30, 2024.
Introduction
My name is Dr Yalemisew Abgaz.
I am based in the School of Computing at Dublin City University. I coordinate Data Literacy and Analytics for 21st Century (CA179) and Introduction to Databases (CA218) modules in the school.
In this portfolio, I describe the design and implementation of the Data Literacy and Analytics for the 21st Century (CA179) module. In response to the initiative by DCU Futures in "Reimagining undergraduate education for an unscripted world", I was entrusted with the task of designing a new module that meets the data literacy requirements of ten DCU Futures programs (Climate & Environmental Sustainability, Psychology and Mathematics, Psychology and Disruptive Technologies, Social Sciences and Cultural Innovation, Bioprocessing , Chemistry with Artificial Intelligence, Mechanical and Sustainability Engineering, Global Challenges, Digital Business & Innovation & BS with Data Analytics, and Physics with Data Analytics ) from the Faculty of Science & Health, Faculty of Engineering & Computing, Faculty of Humanities & Social Sciences, and DCU Business School.
After several consultations with program chairs, industry partners, and other relevant stakeholders, I determined the complex needs of the programs which involve the excellent delivery of core concepts and tools using multiple technologies. Some of the requirements were to build a module, i) that can be delivered in Semester I, Semester II, and Year long in a given academic year, ii) that can be delivered in an online and asynchronous setting where students complete the module anytime during the semester or the academic year, iii) that can be flexible to be delivered as a stand-alone 5-credit module and/or as a series of subtopics embedded in other modules.
To meet these requirements, in line with the DCU Teaching and Learning Strategy 2023-2028 that focuses on Excellence and Flexibility objectives, I designed the module following a Microcurricular as a Service (MaaS) approach which borrows concepts from the Software Engineering domain called Software as a Service (SaaS).
Analogous to SaaS, MaaS focuses on designing a microcurriculum that focuses on a single cohesive and self-contained topic with a single competency and learning outcome, fully-fledged activities, assessments, and microcredits in European Credit Transfer and Accumulation System (ECTS). A module designed as MaaS can be embedded into other modules or can be composed into larger credits (2.5, 5, 7.5, or 10 ECTS) depending on the requirements of the programmes. Figure 1, presents both the structure and contents of the module. The module coordinator (provider) is responsible for the development and management of the MaaS, making it available for clients (students and other module coordinators) to use/reuse the MaaS in their modules.
A microcurrculum contains a single competency and learning outcome with coherent content and resources specifically tailored to achieve the learning outcome. It further contains the corresponding summative assessment which can be administered anytime after the microcurriculum is completed.
Data Literacy and Analytics for the 21st Century is decomposed into seven core topics delivered using 14 microcurricula. The microcurricula are organised into core (4), elective (9), and optional (1) topics. The elective topics are further organised into three routes that allow programs to opt for one of the three technologies widely used in data analytics projects (Spreadsheet, Python, and R). Each microcurriculum is allocated either 0.5, 0.75, or 1.0 ECTS to make a 5-credit stand-alone module (CA179) or embedded into other modules by taking up the allocated credit from the host module.
The implementation
Technically, each Microcurriculum is implemented as a separate Loop page. The module that hosts the microcurricula embeds them as tiles and makes them available to the students according to the module's timetable. Host module coordinators facilitate the learning by directing students to complete the microcurriculum content. All the ready-made content, activities, assessments, and results are provided by the microcurriculum. This significantly reduces the time and resource requirements of the programs.
For programs that opt for a 5-credit module, The four core and three elective microcurricula are delivered as a single module (CA179). For CA179, I provide full support for the students including running orientations, Zoom clinics, and tutorials. Students will take seven assessments corresponding the seven microcurricula and are required to pass the assessment to achieve a passing mark in CA179.
So far, the module has been delivered to 805 students in 2022/23 and 2230 students in the 2023/24 academic year both as a stand-alone 5-credit module or as embedded in a host module. The microcurricula are also embedded into 27 modules in 2022/23 and 20 modules in the 2023/24 academic year.
The MaaS design approach delivers scalable, self-contained, independent, composable, flexibly embedded, and cost effectivel design of modules that are aimed at multiple programs with very large numbers of students. The details of the design philosophy and its implementation can be found in the publication listed below.
- Yalemisew Abgaz; Ciaran Dunne (2023) Pedagogy for Higher Education Large Classes (PHELC) Designing Microcurricula-as-a-service: The case of a large class, cross programme, and online asynchronous module [Link] [DOI]
Teaching Philosophy
Teaching and learning in higher education is a process that involves the construction of knowledge by the learner in collaboration with the educator, environment, colleagues, and the wider society. Teaching involves transferring new knowledge or activating prior knowledge by providing the resources and a conducive environment for the learner to gain the command of themselves, and to reflect and evaluate their learning process and its outcome. My teaching philosophy relies on promoting student-centred active learning that invites the learners to plan and drive their learning by utilising the resources provided to them by the educator. My teaching strategy aligns with the university's strategic objectives: Excellence in inovative curriculum with the focus on excellent, distictive teaching, learning, and assessmnent and Flexibilty in short-form learning and flexible learning pathways for multiple programs.
Methods
My teaching method in a higher education setting is based on applying active learning strategies where students become active participants in the learning process and will take the leadership of their learning journey. I use multiple active learning strategies such as inquiry-based learning (e.g. Alice's case studies in CA179), project-based learning (e.g. Database systems projects in CA218), and problem-based learning. I also use the mastery learning approach to ensure that the learners achieve an acceptable level of knowledge of the subject matter before they move to the advanced topic (e.g. implemented in CA179). My students will see several of these activities embedded in the modules I teach in face-to-face or online sessions. The students are stimulated with multiple means of engagement strategies to participate in active reading, inquiry, peer-learning activities, discussion forums, and practical activities. These approaches allow students to capture and converge their interest in the subject, deeply understand the content, and construct their knowledge individually and in groups. Experiential learning is another strategy I frequently use in delivering programming modules where students directly work on computers to design programs and solutions, discuss challenges, interact with each other and with the teaching assistants and solve inter-disciplinary problems instead of dummy problems. Further, I implement the Universal Design for Learning (UDL) principles in all my modules ensuring multiple means of representation, multiple means of engagement, and multiple means of action and expression. Data Literacy and Analytics for the 21st-century module is designed following these approaches.
Assessment and Feedback
In my teaching practice, I use assessment for learning, as learning, and of learning whenever appropriate. The assessment strategy I implemented for the Data Literacy and Analytics module relies on assessment for learning and assessment as learning. Several formative assessment questions are integrated into the module in the form of knowledge check questions, peer learning activities, and discussion forums. Feedback is an integral part of the formative assessment pieces where students receive feedback with pointers referring to the original content along with supplementary resources that are related to the topics covered in the assessment. Additional drop-in clinics are also organised to address specific issues encountered by students in their formative assessment.
Summative questions are used as an assessment of learning at the end of each microcurricula. To ensure the engagement and the quality of the learning, at the end of each microcurriculum, students are presented with summative assessment questions. I used a novel approach to the summative assessment in that students can take the assessment at any time as soon as they complete the topic, without waiting for the end of the semester. Students are also allowed to retry the assessment questions until they achieve the passing mark which is currently 80% or above. To ensure the robustness and integrity of the assessment, I used a large set of questions for each topic of a microcurriculum and systematically generated the questions ensuring that students got different sets of questions at any given time in a semester.
Currently, I have proposed an approach to further enhance the assessment using the co-creation of formative and summative assessments together with a selected number of students with a rationale of generating representative questions that improve the areas of assessment, and reduce the recycling of existing questions. This further allows the module to have multiple means of engagement by including student's perspectives. The implementation is planned at the end of the semester and will be integrated into the module in the 2024/25 academic year.
Student support
My approach to supporting students begins with understanding the requirements of the students. First, students with any special needs are identified from the GURU system. All the resources are designed following UDL principles. Further, I provide different levels of support for the students. There are dedicated Zoom Drop-in Clinics to answer any questions arising from students. Support is also available to students via email and chat. For the stand-alone 5-credit modules, an additional three face-to-face tutorials are organised. With the support of teaching assistants, we also provide continuous feedback to the discussion forums and peer learning activities of the students. Although it is extremely difficult to provide feedback for all students, we are engaged with the majority of the peer-learning activities and discussion forums. I also give general feedback when applicable.
Teaching Development and Personal Reflection
To equip my teaching practice with state-of-the-art theory and practice in education, I am engaged in several short-term and long-term training courses. I frequently seek feedback from students, colleagues, and the literature and reflect to improve or change my practice. Further to that, some of the completed and ongoing courses and training are listed below.
- Introduction to ABC Learning Design by TEU, DCU
- Graphic Facilitation by the Graphic Change Academy
- The UDL Badge from the National Forum
- The UDL Facilitator Badge from the National Forum
- In progress: Masters of Art in Educational Practice course.
I also actively participated in the weekly Pebbles session organised by the School of Computing and delivered presentations related to my microcurricula design, novel methods of assessment, and UDL.
Summary of Events in 2023/2024
Major events
- December 2023: Strategic Alliance for Teaching, Learning and Evaluation (SATLE) funding awarded in December 2023. Kickoff in 2024. Amount: €5,000
- January 2024: A high level of engagement is observed from Bioprocessing students with a 100% pass rate.
- January 2024: A very large volume of discussion forums and peer learning activities in Semester II.
- December 2023: Completion of UDL Badge and Facilitator course from the National Forum.
- September 2023: Three additional programs joined for the 5-credit module in semester II.
- September 2023: Interviewed by DCU Futures Evaluation Team (Dr Hadger Taibi).
Do you know?
Students surveyed so far indicated that the most appreciated features of CA179 are the content, the flexibility, and the assessment. Going through diversified content, with flexible timing and being able to take the assessment at their convenient time after completing a topic are the most appreciated features by several students. Students further appreciated the multiple options they had to re-attempt the assessment. This gave them the option of mastering the content. I have observed some students striving to score the highest mark despite the passing mark they attained on their first attempt. Some of the feedback is included below.
Content
"I didn't know that data literacy is defined as the skill to use and utilise data. I'm looking forward to using the data of my choice to utilise data that I think necessary to know in day to day life and hopefully this data use can make the difference in businesses."
"I came across the enumerate function in python3 over the Summer and was happy to get clarification on what enumerated data types are. I am excited to use this function along with other dictionary functions in the future."
"I wasn't aware on how many different aspects there is to data literacy and am excited to learn more about the DIKW pyramid."
Engagement Format
“I liked the videos it’s nice to have information presented in different ways.”
“The layout helped me to learn the information effectively, as I was switching from visual learning to doing activities throughout the courses.”
“The visual representation of information and data”
“I found the learning activities helpful to solidify the knowledge I learnt in the module.”
Assessment
“The tests at the end were good I like the mix of media, so it didn’t feel like I was reading a massive amount of text but had visuals and tasks to break it up.”
“I found the questions within the learning activities helpful to solidify my knowledge.”
“Multiple choice throughout to stay engaged”.
Summary of Events in 2022/2023
Major events
- August 2023: Successful completion of resit for Data Literacy and Analytics Module in August 2023.
- May 2023: Successful completion of the first cycle of Data Literacy and Analytics Module in May 2023.
- June 2023: Our first paper on Microcurriculum is published and presented at PHELC 2023 Symposium.
- December 2023: Full implementation of Introduction to statistics in collaboration with School of Mathematics ( Professor Brien Nolan and Sarah Jane Mc Glinchey).
- December 2022: implementation of Python and R routes on Loop. All the 14 topics completed.
- December 2022: A very high-rate of successful completion of the 5-credit stand-alone module.
- September 2022: A high-level of engagement of module coordinators from the 10 DCU Futures programs.
- September 2022: Kick-off! Successful implementation of the 7 Loop pages of the module in September and enrolment of 864 students.
- July/August 2022: Interview of six DCU academics and 4 experts from industry. The content is embedded in the DLATs (Professor Alan Smeaton, Dr Suzanne Little, Dr Andrew McCarren, Dr Marta Rocchi, Dr Michael Scriney, Dr Greg Foley, Dr Anthony McCoy, Mr Simon McGarr, Dr Niall Browne, Mr Michael Kennedy, and Dr Graham Healy).
- July 2022: Development of the draft module content and the recording of introductory videos in collaboration with DCU Studio (Matthew Waters, Caitríona Nic Giolla Mhichíl )
- May 2022: Data Literacy and Analytics Expert group delivered the competency statement and learning outcomes (Several members from DCU).
- April 2022: Workshops with Prof Dave Goldberg. Culture and Contraries and Spirits that Transcend Traditional Curriculum Innovation.
- January - March 2022: Eight consultation meetings and additional eight validation meetings held with the ten DCU Futures program chairs. Additional two consultation meetings with School of Computing academics (Dr Karsten Fleischer, Dr Stephen Power, Dr Lorna Fitzsimons, Dr Louise Hopper, Dr Darren Clarke, Dr Loanda Cumba, Dr Lorraine Boran, Mr. Gerry Conyngham, Dr Greg Foley, Dr Shirley Coyle, Dr Emma Coyle).
- January - December 2022: Weekly Data Literacy and Analytics design meetings with Lisa Whelan and Dr Ciaran Dunne.
- November 2021: The first workshop on Data Literacy and analytics held and the deliberation of a working definition of Data Literacy and Analytics was reached (Organised by Lisa Whelan, Dr Ciaran Dunne, Prof. Blanaid White)