This research uncovers the validity, reliability, level of difficulty, and discrimination power of an artificial intelligence (AI) generated of Economics items for secondary students. Items can be prepared by either the teacher (teacher-made), examination bodies (standardised) or now AI-bred tests. Students’ responses to AI-generated items were used in this research. A sample of 1,036 students was selected using a random sampling technique. The instrument used for the study was the AI Economics item. The validity and reliability of the instrument were conducted using content and face validity methods, while the Kuder Richardson 20 (Kr-20) method obtained a coefficient that was determined to be 0.76. Data were analysed using R-programming (coding) for p-value and d-value analysis. The rules of thumb used for the judgment on items analysis were P-values Indies from 0. 40 to 0.60 = appropriate (easy), 0.00 – 0.39 and < 0.60 = inappropriate (very difficult and too easy), while d-values indices from 0. 50 to 0.1 = Good, Indies < 50 and all negative indices = Bad. Results revealed that 22 items (73%) out of the 30 items were appropriately difficulty level (p-value), while eight items (27%) out of the 30 items were inappropriate difficulty (p-value). Also, from the computed d-values, seven items (23%) (30,27,22,12,6,5,2) were bad discriminators, while 23 items (77%) were good discriminators. This implies that Economics items generated by ChatGPT AI are appropriate and good in terms of their difficulty level and discrimination power. Based on the findings, it was concluded that ChatGPT’s AI-made Economics items are reasonably appropriate in accuracy and satisfactory regarding the p-value and d-value indices. Based on these recommendations, test constructors should use ChatGPT or other AI in test preparation, administration, and scoring. Also, Economics students should use ChatGPT’s AI as a practice tool when preparing for their test.
Keywords: Artificial intelligence for e-commerce items, Psychometrics properties of ChatGPT-bred items, Item difficulty and discrimination Power.
1Udemba, Esther Chinenye, Jacob Esu Odiong Oluwayemisi Damilola Akomolafe
PSYCHOMETRICS PROPERTIES OF ARTIFICIAL INTELLIGENCE (CHATGPT) BREED ECONOMICS MULTIPLE CHOICE ITEMS
This research uncovers the validity, reliability, level of difficulty, and discrimination power of an artificial intelligence (AI) generated of Economics items for secondary students. Items can be prepared by either the teacher (teacher-made), examination bodies (standardised) or now AI-bred tests. Students’ responses to AI-generated items were used in this research. A sample of 1,036 students was selected using a random sampling technique. The instrument used for the study was the AI Economics item. The validity and reliability of the instrument were conducted using content and face validity methods, while the Kuder Richardson 20 (Kr-20) method obtained a coefficient that was determined to be 0.76. Data were analysed using R-programming (coding) for p-value and d-value analysis. The rules of thumb used for the judgment on items analysis were P-values Indies from 0. 40 to 0.60 = appropriate (easy), 0.00 – 0.39 and < 0.60 = inappropriate (very difficult and too easy), while d-values indices from 0. 50 to 0.1 = Good, Indies < 50 and all negative indices = Bad. Results revealed that 22 items (73%) out of the 30 items were appropriately difficulty level (p-value), while eight items (27%) out of the 30 items were inappropriate difficulty (p-value). Also, from the computed d-values, seven items (23%) (30,27,22,12,6,5,2) were bad discriminators, while 23 items (77%) were good discriminators. This implies that Economics items generated by ChatGPT AI are appropriate and good in terms of their difficulty level and discrimination power. Based on the findings, it was concluded that ChatGPT’s AI-made Economics items are reasonably appropriate in accuracy and satisfactory regarding the p-value and d-value indices. Based on these recommendations, test constructors should use ChatGPT or other AI in test preparation, administration, and scoring. Also, Economics students should use ChatGPT’s AI as a practice tool when preparing for their test. Keywords: Artificial intelligence for e-commerce items, Psychometrics properties of ChatGPT-bred items, Item difficulty and discrimination Power.