This post is co-authored by Behzad Fatahi and Roger Hadgraft.
In Autumn 2023, we sought to reinvigorate our Soil Behaviour subject, part of the UTS Civil Engineering course. We wanted to elevate students’ capacities to engage with and critically dissect civil engineering challenges. By introducing ChatGPT, we not only aimed to deepen students’ understanding of soil behaviour but also to make the learning experience more enjoyable. Indeed, a recent roundtable panel discussion at UTS and TEQSA/CRADLE panel discussion emphasised the importance of critically engaging with generative AI (GenAI).
We also wanted to bring some fun and novelty to the assessment landscape. This innovative approach provided the opportunity for students to engage directly with GenAI, extracting, analysing, and critically evaluating its responses.
The nuances of the innovative assessment
The Research Project task attracts 30% of the overall mark of the subject. Students could choose one of three options:
- In Option 1, students researched soil behaviour concepts using classical methods, presenting their findings via literature review, data, images, and case studies.
- Option 2 adopted computational simulation of building foundations and excavations using industry standard software. Students had regular hands-on computer lab sessions and used the Bring Your Own Device (BYOD) facility in FEIT via The WorkSpace portal.
- The new choice, Option 3, incorporated ChatGPT; students compared its responses to established soil behaviour knowledge sources and also contrasted its predictions against the software simulation results (Option 2). This unique blend of traditional and AI-based methods aimed to gauge students’ analytical engagement across different educational approaches.
Observational insights, and enriched student experience
Students were assessed on the three task options using a rubric, with the most significant item being the ability to engage and critically analyse (40%). Statistical data from this assessment showed that students who engaged with the GenAI model, ChatGPT, performed best on average in critical engagement (85% on average, compared to 67% and 78% for Options 1 and 2).
Many students took multiple tries (usually 4-7 attempts) to ask ChatGPT a good question. These students could also spot when the AI’s answer was right or wrong. On the flip side, some students had trouble asking specific questions and just accepted the AI’s answers without really questioning or understanding them.
Samples of anonymous feedback from students reflected a largely positive reception to this novel pedagogical approach.
I particularly liked the ChatGPT task. It was a new type of assessment format and allowed me to critically analyse the AI tool. I was able to find that the tool didn’t go much in detail but rather just skimmed over the question and blabbered on. Tasks like these are really interesting, and I hope to see such tasks in the future.
Feedback from a student
The path forward
While this was our inaugural venture with a limited cohort of 134 students in Soil Behaviour subject involved, based on our current observations, we recommend other educators, especially those in the engineering field, to consider exploring and integrating Generative AI tools into their subjects as an enhancement to traditional teaching methods. This not only can enrich the learning experience but can also hone students’ critical thinking skills.
However, it is essential to provide clear guidelines on how to interact with these tools effectively. Workshops or tutorials focused on formulating precise queries can help students maximise their interactions with AI. Additionally, educators should emphasise the importance of cross-referencing AI-generated information with authoritative sources, ensuring students remain discerning and do not take AI responses at face value. This blended approach could offer a dynamic, engaging, and rigorous learning environment for the digital age.
The teaching team is now exploring opportunities to employ other Generative AI platforms such as Midjourney to enrich other available learning experience in the subject, such as data interpretation and reporting for soil testing laboratories, Geological practicals and drawings.
Detailed explanation and analysis of this initiative will be presented later this year in the 34th Australasian Association for Engineering Education Annual Conference in Gold Coast.