Coherense · Kratos · Research Version

AI Tool Ethical Evaluation

Based on the ECET AI Framework · Grotewold, Warren & Beck (2024)

What is Kratos?

Kratos is a structured evaluation instrument for educators considering an AI tool for classroom use. It operationalizes the ECET AI Framework (Ethical Choices with Educational Technology for Artificial Intelligence), developed by Grotewold, Warren, and Beck (2024) and grounded in international AI guidelines from the EU, UNESCO, SIAA, the US Department of Education, the Department of Defense, and the White House.

The framework evaluates any AI tool across three sequential bands: whether the tool supports your learning goals (Idea), whether it can realistically be implemented (Feasibility), and whether using it is ethically sound (Ethics). Each band contains six items drawn directly from the ECET AI principles published in Grotewold, Warren, and Beck (2024).

This research version includes a usability feedback section so your experience with the instrument informs its ongoing development. Items labeled ECET AI are drawn directly from the published framework. Items labeled Kratos Extension are additional dimensions added for this research instrument.

Time required
30–60 minutes total. Most time is spent researching the tool before scoring. Use Save progress to return later.
Scoring
Rate each of 18 items on a 1–10 scale. Your average score updates live. Each item includes specific research guidance.
What you need first
Hands-on access to the tool, its privacy policy, and at least 30 minutes of direct use. Do not score from marketing materials.
Submission
Download CSV and HTML report files, then email both to the research team. The email button pre-fills everything.

How to complete this evaluation — step by step

1
Read this introduction before you begin.
The ECET AI Framework is designed to be completed in order — Idea first, then Feasibility, then Ethics. Understanding the sequence will improve your answers.
2
Get hands-on access and spend 30 minutes with the tool.
Create a free account or trial. Use the tool as your students would for a realistic activity. You cannot accurately score without this direct experience.
3
Research the privacy policy and educator documentation.
Look up FERPA compliance statements, data retention and deletion policies, and whether student-submitted content trains the AI model. Search: [tool name] "privacy policy" FERPA education
4
Work through all 18 ECET AI items in order.
Read each item's "How to research this" note before scoring. Complete all Idea items before moving to Feasibility. Complete Feasibility before Ethics.
5
Complete the usability feedback section (Step 5).
This section is about Kratos, not the tool you evaluated. Your honest feedback — including frustrations — directly informs the instrument's improvement.
6
Download both files and email them to the research team.
In Step 6, download the CSV data file and the HTML report, then use the pre-filled email button to submit both to scott.warren@unt.edu.

Score interpretation

7 – 10
Meets or exceeds standard
4 – 6
Partially meets standard
1 – 3
Does not meet standard

The average score across all 18 items is the primary quantitative outcome. Band averages (Idea, Feasibility, Ethics) are reported separately. Your notes for each item are equally important for qualitative analysis.

Step 1 of 6

Informed Consent

Before entering any information, please read and complete the official informed consent form.

Study information

Purpose

This study examines how pre-service teachers and higher education faculty evaluate AI tools using the ECET AI Framework (Grotewold, Warren, & Beck, 2024). The purpose is to assess the usability and effectiveness of the Kratos instrument and to gather structured evaluations of AI tools across pedagogical, feasibility, and ethical dimensions.

What you will do

You will identify an AI tool you have used or are considering for classroom use, research it using provided guidance, and rate it across 18 ECET AI items on a 1–10 scale. You will also provide written notes and complete a usability feedback section about the Kratos instrument itself.

Time commitment

Approximately 30–60 minutes. You may save your progress and return later. A follow-up focus group or interview may be requested separately.

Risks and benefits

Minimal risk — similar to ordinary professional reflection. There is no guaranteed personal benefit, though your feedback directly informs instrument revision and framework validation.

Confidentiality

Data will be stored in secure, password-protected files or university-approved cloud storage. Reports will use de-identified summaries unless you explicitly consent to attribution. Records will be retained for at least three years past the end of the research.

Voluntary participation

Participation is voluntary. You may withdraw at any time without penalty and may skip any question that makes you uncomfortable.

Contact

Questions about the study: Dr. Scott J. Warren, scott.warren@unt.edu, 940-369-7489. Questions about your rights as a research subject: UNT Office of Research Integrity and Compliance, 940-565-4643, untirb@unt.edu.

Complete the informed consent form

This study's informed consent record is collected through UNT's secure Qualtrics system, separate from this evaluation site. Please:

1
Click the button below to open the consent form in a new tab.
Read it carefully and respond to indicate whether you agree to participate.
2
Return to this tab once you have submitted the form.
This browser tab will stay open on this page while the form opens separately.
3
Check the confirmation box below and continue.
You'll then move on to entering your demographic information.
Please open and complete the informed consent form, then check the box above to continue.
Step 2 of 6

About You

A few questions about your background before the evaluation begins.

About you

Please enter your first name to continue.
Step 3 of 6

Tool Information

Identify the AI tool you will evaluate and describe your intended use.

Before you continue: you must have hands-on experience with this tool. Do not evaluate from marketing materials or secondhand descriptions. If you have not yet used the tool, pause here, create a free account or trial, and spend at least 30 minutes using it as your students would. Then return.

About the tool

Preparation checklist — complete before Step 4

Your ECET AI scores will only be accurate if you have done this research first.

1
Get hands-on access. Create a free account or open the tool at its website. Use it as your students would for at least 30 minutes for an activity similar to what you plan to assign.
2
Read the privacy policy and terms of service. Look for FERPA compliance, data retention and deletion policy, and whether student-submitted content trains the AI model. Search: [tool name] "privacy policy" FERPA education
3
Find independent educator reviews. Look for Common Sense Media ratings, EdSurge reviews, or peer-reviewed studies. Search: [tool name] "educator review" OR "classroom use" OR "student data concerns"
4
Note what evidence the vendor provides. Look for peer-reviewed research or independent studies the vendor cites. Note whether claims come from independent research or internal marketing materials only.
Please enter the tool name and describe your intended use to continue.
Step 4 of 6 — ECET AI Framework · Grotewold, Warren & Beck (2024)

ECET AI Review

Rate all 18 items in order. Read each "How to research this" note before scoring. Your average updates live.
Idea
—
Feasibility
—
Ethics
—
Overall avg
—
0 of 18 rated

Band open feedback

After completing all 18 items, provide a brief written summary for each band.

Idea Band

Feasibility Band

Ethics Band

Overall assessment

Step 5 of 6 — Kratos Research

Usability & UX Feedback

This section is about Kratos itself — not the AI tool you evaluated. Your honest responses directly inform the instrument's revision.

Important: There are no right or wrong answers here. Critical and candid feedback is more valuable than positive feedback for improving this instrument. Please answer based on your actual experience.

Time and effort

Directions and navigation

The ECET AI questions

The 1–10 rating scale

Interface and design

Value and future use

Step 6 of 6

Review & Submit

Review your evaluation summary, then download both files and email them to the research team.

Download your files

Generate both files. Attach both to the same email submission.

1
Click Download CSV — your complete structured data for analysis.
2
Click Download HTML Report — formatted evaluation summary.
3
Click Open email client to send both files to scott.warren@unt.edu.

Email to research team

Send both downloaded files to scott.warren@unt.edu. The button below pre-fills the subject and body.

Kratos Evaluation — [Your Name] — [Tool Name]
Thank you for completing this evaluation. Your feedback directly informs the revision of Kratos and the ongoing development of the ECET AI Framework (Grotewold, Warren & Beck, 2024).