AI Literacy Research
Executive Summary
Updated with follow-on Parent and Teen research:
- 62% of the parents and teens read all or most of the five learning blocks
- 12% of the parents and teens read some of the learning blocks
- 26% of the parents decided against participating.
We are continuing to research Parent Guide usage and satisfaction levels through surveys of parents, teachers and teens in 2021. While satisfaction levels of parents that used the Guide in the recommended manner – parents and their teens go through the Guide and view the videos together – were high (see the Parents section below), only 23% of the 107 parent respondents actually did so. Instead, 41% of the parents first read the content, next asked their teens to read and the content and watch the videos independently, and then asked teens the discussion questions. This required follow-up research which we conducted in January and February and is in the new Parents and Teen section. 26% of the parents declined to participate because (a) of their lack of confidence or comfort with the subject matter (17%), (b) they did not have time (5%), or (c) they felt that it wasn’t necessary because the material was covered in school or for another reason (4%).
Background
AI Literacy is the first website for parents to guide their teens to videos that show how AI is increasing affecting the world around them. It uses the proven methodology of “learning blocks” to divide topics into digestible and easy-to-understand modules. The learning blocks and videos work together to help increase teens’ critical thinking skills and teaches them how to leverage AI so that they can make discerning educational, career and life decisions. Parents and teens require no coding or special technical knowledge and the Guide does not teach coding or how to develop AI products. Instead, it explains how AI is affecting our world and will increasingly impact all of us. An alpha version of the site was launched in 2019 and portions of the content used in the first learning blocks were tested through a series of articles in the Family Online Safety Institute’s website (Example: When Is Seeing No Longer Believing?). Feedback was crowdsourced from the articles and then the learning block and video methodology was tested with parents, teachers and students in a FERPA and GDPR compliant manner.
Parents
The Parents section of this research reflects the feedback from the 23% of parents that instructed their teens in the manner we contemplated – by sitting with their teen and going through the Guide together.
Initial Parents Research Results:
- Learning block 1: Getting to know AI – 93% of parents were comfortable that they understood most of it, 89% were comfortable when they explained it to their 13–17-year-old children, and 94% felt comfortable that their children understood it.
- Learning block 2: When is seeing no longer believing – 94% of parents were comfortable that they understood most of it, 92% were comfortable explaining it to their 13-17-year-old children, and 93% felt comfortable that their children understood it.
- Learning block 3: Don’t let deepfakes fake you out – 91% of parents were comfortable that they understood most of it, 89% were comfortable explaining it to their 13-17-year-old children, and 92% were comfortable that their children understood it.
- Learning block 4: Ethical AI – why the thoughtful development of AI is so important – 82% of parents were comfortable that they understood most of it, 79% were comfortable explaining it to their 13-17-year-old children, and 84% of them were comfortable that their children understood it.
- Learning block 5: AI for Sustainability and for Good – this uses more than 50 videos to explain how AI is helping the world meet the UN’s 17 SDGs, 85% of parents were comfortable that they understood most of it, 80% were comfortable explaining it to their 13-17 year-old children, and 81% that their children understood most of it.
Additional findings:
- Prior to reading the AI Literacy content, 7% of the parents said that they had a sufficient understanding about AI to have a general conversation with their teens about it.
- Parents were comprised of mothers (68%), fathers (21%), and other (11%). Other was a combination of grandparents and older siblings.
- It is significant to note that in most cases, parents felt that their teens understood the concepts better than they did.
- When asked what additional areas should be covered, parents asked for a learning block about how AI will affect jobs and careers in the future (which is in draft form). Parents wanted a web-based worksheet so that they could assess their teen’s level of comprehension. Parents also asked for additional resources in each learning block that they could choose from to further engage their teen (e.g., more videos, games and hands-on experiments to help explain, demonstrate and reinforce each learning objective), which we added in 2020 from the available free online resources that we identified.
Teachers
AI Literacy is a resource designed to help facilitate a conversation about AI in the classroom for teachers that do not have a technical background and where a introductory class on AI hasn’t been established. Research suggests that in the US, only 10% of K12 teachers feel confident about teaching technology in general. It is likely that fewer would be confident teaching AI. Conversations with teachers and administrators indicated that AI can be taught by computer science, math, business, science, social science and vocational education departments. The following summarizes the results of a survey conducted in 2020 with secondary school teachers that said teaching AI literacy was relevant to their discipline.
Pre-survey Teacher Responses
- 12% of the teachers, on average, said that they had the existing skillset to design lesson plans or teach either a quarter or semester long course about a non-technical course about AI that included (a) how AI is being used in everyday life, (b) AI ethics, (c) AI myths and facts, (d) misinformation and deepfakes, and (e) how AI will affect jobs and careers in the future.
- 21% of the teachers, on average, said that they had the existing skillset to design lesson plans or teach one to five 50-minute non-technical classes about AI that included (a) how AI is being used in everyday life, (b) AI ethics, (c) AI myths and facts, (d) misinformation and deepfakes, and (e) how AI will affect jobs and careers in the future.
Teacher Survey Results
However, after reading the five learning blocks in the Parent’s Guide plus a draft of the next learning block (how AI will affect jobs and careers in the future):
- 94% of the teachers said that using the learning blocks as a guide they could teach a 50-minute class in (a) how AI is being used in everyday life, (b) AI myths and facts, and (c) misinformation and deepfakes
- 93% of the teachers said that using the AI ethics learning block as a guide, they could teach a 50-minute class on AI ethics.
- 91% of the teachers said that using the draft learning block about how AI will affect jobs and careers in the future, they could teach a 50-minute class on jobs and careers.
- 90% of the teachers said that using the learning blocks as a guide, they could teach a 50-minute class showing how AI was helping meet the UN’s 17 Sustainable Development Goals.
Teacher Promoter and Detractor Results
When reviewing the “good practices” resources repository on the AI Literacy site, teachers rated it as follows:
- 73% – very helpful
- 15% – helpful
- 4% – neither helpful nor not helpful
- 7% – not very helpful
- 1% – not at all helpful
Detractors criticized a lack of precision when using the site search and that the Editor’s Comments weren’t sufficiently prominent. The Editor’s Comments problem is in development and will be corrected in March. Teachers also asked why coding wasn’t included when teaching teens about AI literacy. We believe that learning how to code is very important and useful. However, the primary target market of this site is an international audience of parents and integrating coding in the Guide would significantly reduce parent uptake and engagement. We do offer coding and data science resources in the Good Practices and Resources repository.
Parents and Teens Update
Background
While the Parent’s Guide was developed to be “parent friendly” and to be used by parents to actively show and guide their teens, only 23% of parents in the survey did so in the manner that we contemplated. In January and February 2021, additional research was conducted with the 41% of the parent universe that read the content first, asked their teens to read and watch the content independently and then followed up by asking teens the discussion questions. Each parent represented one family. Of the 44 parents/families in this group, 37 parents/families and their 51 teens participated in the follow-up research. The primary purpose of this research was to understand the effectiveness of the Parent’s Guide with this segment of parents, since we didn’t contemplate that the Guide would be used in this manner and consequently, it was beyond the scope of the questions that we asked in the original survey.
Parent Research Results
In this study, of the 37 parents/families said that their teen(s) started the course, 26 (70%) of the parents/families said that their 39 teen(s) completed all or almost all of the course and 11 (30%) said that their 12 teen(s) completed some of the course but not all or almost all of the course. When the 26 parents/families in the first group were asked the main reason their teen(s) finished the course, 15 said it was a combination of them actively engaging with their teen(s) and the quality of the Guide, 9 said it was primarily because they actively engaged their teen(s) and 2 said it was primarily due to the quality of the Guide. When the 11 parents/families in the second group were asked the main reason their teen(s) didn’t finish most of the course, 8 said it was primarily because they didn’t actively engage their teen(s) and 3 said it was a combination of them not actively engaging their teen(s) and the quality of the guide.
Analysis of Parent Research
93% of the parents having more two teens take the course completed all or most of the program, while 50% of the parents having only one teen completed all or most of the program. Consequently, having more than one teen take the course at the same time may significantly improve completion rates. 70% of the teens that read and watched the Guide independently completed all or almost all of the course as compared to 100% of the teens in the initial research that sat with their parents and went through the Guide together. Consequently, Guide completion rates appear to be strongly influenced by the teen sitting with a parent or working independently.
Teen Research Results
While some of the teens believed that they completed more of the course than their parents, this was not material to this research’s focus. Teen satisfaction with the Guide was mixed and a summary follows:
- Quality of video content – 94% of the teens rated it good or excellent
- Unaided comments – “interesting”, “really important”, “liked the positivity” and “learned something new”
- Look and feel – only 11% of the teens rated the look and feel of the Parent’s Guide as good (no one rated it as excellent)
- Unaided comments – “boring”, “too much like school”, “basic”, “old school”
- Length of videos – 91% of the teen rated the length to be good or excellent
- Unaided comments – some “videos were too short”…that they were only relevant when viewed “with the others” and “add longer videos” (so I can) “dig deeper” or “watch more if it’s interesting”, “like that there is a vid(eo) selection so I could x-out ones I didn’t like, like on Youtube”, “Liked a mix of long and short videos”
- Learning block topics relevancy – 87% of the teens rated the relevancy of the existing topics good or excellent
- Unaided comments about content that should be added – “how AI will affect my career/future”, “how to get into AI”, the “best free places to learn to code”, “the singularity/Skynet/killer robots/killer drones” risks and dangers
Analysis of Teen Research
These results were unexpected. The Guide had been designed to engage the parent because we anticipated that the parent would oversee and manage their teen(s)’ study of the content. We did not contemplate that many teens would read the Parent’s Guide independently rather than together with a parent and consequently, the design of the Parent’s Guide has a clean and minimalistic parent-friendly “look and feel”. Because many of the teens read and watched the content independently, the parent-friendly design disengaged the teens and likely impacted retention and completion rates. Unfortunately, we did not ask this question to teens in the survey. Further, while 100% of the teens completed the Guide when sitting with their parents and going through the Guide together, only 70% of the teens that read the Guide independently completed all or most of the course. Also, it’s reasonable to assume that a number of the 7 parents that declined to participate in the second survey did not complete all or most of the course and that the actual completion rate in the group where teens read the Guide independently may have been less than 70%. These results suggested that we (a) develop an engaging teen-friendly Guide, (b) address other suggestions raised by the teens and (c) identify tactics to increase the number parents that sit with their teen(s) and go through the Guide together.
Progress and Next Steps
Based upon this most recent research, we have added a separate A Student’s Guide to Understanding AI to the site and are developing a graphically intensive version with a student-engaging “look and feel” which will be added to the site in March. We have also updated the videos for each of the SDGs in Learning Block 5 so that most have a mix of both 1-3-minute and 4+ minute videos. Next, we will explore tactics that (a) reduce the percentage of parents opting out of this program and (b) increase the number of parents that sit with their teen(s) to go through the Guide together and thus. increase the percentage of teens that complete the program.