Automated Online Exam Proctoring

Yousef Atoum, Liping Chen, Alex X. Liu, Stephen D.H. Hsu, Xiaoming Liu

Research output: Contribution to journalArticle

  • 2 Citations

Abstract

Massive open online courses and other forms of remote education continue to increase in popularity and reach. The ability to efficiently proctor remote online examinations is an important limiting factor to the scalability of this next stage in education. Presently, human proctoring is the most common approach of evaluation, by either requiring the test taker to visit an examination center, or by monitoring them visually and acoustically during exams via a webcam. However, such methods are labor intensive and costly. In this paper, we present a multimedia analytics system that performs automatic online exam proctoring. The system hardware includes one webcam, one wearcam, and a microphone for the purpose of monitoring the visual and acoustic environment of the testing location. The system includes six basic components that continuously estimate the key behavior cues: user verification, text detection, voice detection, active window detection, gaze estimation, and phone detection. By combining the continuous estimation components, and applying a temporal sliding window, we design higher level features to classify whether the test taker is cheating at any moment during the exam. To evaluate our proposed system, we collect multimedia (audio and visual) data from 24 subjects performing various types of cheating while taking online exams. Extensive experimental results demonstrate the accuracy, robustness, and efficiency of our online exam proctoring system.

LanguageEnglish (US)
Article number7828141
Pages1609-1624
Number of pages16
JournalIEEE Transactions on Multimedia
Volume19
Issue number7
DOIs
StatePublished - Jul 1 2017

Profile

Education
Monitoring
Microphones
Scalability
Acoustics
Personnel
Hardware
Testing

Keywords

  • Covariance feature
  • gaze estimation
  • online exam proctoring (OEP)
  • phone detection
  • speech detection
  • text detection
  • user verification

ASJC Scopus subject areas

  • Signal Processing
  • Media Technology
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

Automated Online Exam Proctoring. / Atoum, Yousef; Chen, Liping; Liu, Alex X.; Hsu, Stephen D.H.; Liu, Xiaoming.

In: IEEE Transactions on Multimedia, Vol. 19, No. 7, 7828141, 01.07.2017, p. 1609-1624.

Research output: Contribution to journalArticle

Atoum Y, Chen L, Liu AX, Hsu SDH, Liu X. Automated Online Exam Proctoring. IEEE Transactions on Multimedia. 2017 Jul 1;19(7):1609-1624. 7828141. Available from, DOI: 10.1109/TMM.2017.2656064
Atoum, Yousef ; Chen, Liping ; Liu, Alex X. ; Hsu, Stephen D.H. ; Liu, Xiaoming. / Automated Online Exam Proctoring. In: IEEE Transactions on Multimedia. 2017 ; Vol. 19, No. 7. pp. 1609-1624
@article{159cadc63c2b47fcb11b54373b6e6e2c,
title = "Automated Online Exam Proctoring",
abstract = "Massive open online courses and other forms of remote education continue to increase in popularity and reach. The ability to efficiently proctor remote online examinations is an important limiting factor to the scalability of this next stage in education. Presently, human proctoring is the most common approach of evaluation, by either requiring the test taker to visit an examination center, or by monitoring them visually and acoustically during exams via a webcam. However, such methods are labor intensive and costly. In this paper, we present a multimedia analytics system that performs automatic online exam proctoring. The system hardware includes one webcam, one wearcam, and a microphone for the purpose of monitoring the visual and acoustic environment of the testing location. The system includes six basic components that continuously estimate the key behavior cues: user verification, text detection, voice detection, active window detection, gaze estimation, and phone detection. By combining the continuous estimation components, and applying a temporal sliding window, we design higher level features to classify whether the test taker is cheating at any moment during the exam. To evaluate our proposed system, we collect multimedia (audio and visual) data from 24 subjects performing various types of cheating while taking online exams. Extensive experimental results demonstrate the accuracy, robustness, and efficiency of our online exam proctoring system.",
keywords = "Covariance feature, gaze estimation, online exam proctoring (OEP), phone detection, speech detection, text detection, user verification",
author = "Yousef Atoum and Liping Chen and Liu, {Alex X.} and Hsu, {Stephen D.H.} and Xiaoming Liu",
year = "2017",
month = "7",
day = "1",
doi = "10.1109/TMM.2017.2656064",
language = "English (US)",
volume = "19",
pages = "1609--1624",
journal = "IEEE Transactions on Multimedia",
issn = "1520-9210",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "7",

}

TY - JOUR

T1 - Automated Online Exam Proctoring

AU - Atoum,Yousef

AU - Chen,Liping

AU - Liu,Alex X.

AU - Hsu,Stephen D.H.

AU - Liu,Xiaoming

PY - 2017/7/1

Y1 - 2017/7/1

N2 - Massive open online courses and other forms of remote education continue to increase in popularity and reach. The ability to efficiently proctor remote online examinations is an important limiting factor to the scalability of this next stage in education. Presently, human proctoring is the most common approach of evaluation, by either requiring the test taker to visit an examination center, or by monitoring them visually and acoustically during exams via a webcam. However, such methods are labor intensive and costly. In this paper, we present a multimedia analytics system that performs automatic online exam proctoring. The system hardware includes one webcam, one wearcam, and a microphone for the purpose of monitoring the visual and acoustic environment of the testing location. The system includes six basic components that continuously estimate the key behavior cues: user verification, text detection, voice detection, active window detection, gaze estimation, and phone detection. By combining the continuous estimation components, and applying a temporal sliding window, we design higher level features to classify whether the test taker is cheating at any moment during the exam. To evaluate our proposed system, we collect multimedia (audio and visual) data from 24 subjects performing various types of cheating while taking online exams. Extensive experimental results demonstrate the accuracy, robustness, and efficiency of our online exam proctoring system.

AB - Massive open online courses and other forms of remote education continue to increase in popularity and reach. The ability to efficiently proctor remote online examinations is an important limiting factor to the scalability of this next stage in education. Presently, human proctoring is the most common approach of evaluation, by either requiring the test taker to visit an examination center, or by monitoring them visually and acoustically during exams via a webcam. However, such methods are labor intensive and costly. In this paper, we present a multimedia analytics system that performs automatic online exam proctoring. The system hardware includes one webcam, one wearcam, and a microphone for the purpose of monitoring the visual and acoustic environment of the testing location. The system includes six basic components that continuously estimate the key behavior cues: user verification, text detection, voice detection, active window detection, gaze estimation, and phone detection. By combining the continuous estimation components, and applying a temporal sliding window, we design higher level features to classify whether the test taker is cheating at any moment during the exam. To evaluate our proposed system, we collect multimedia (audio and visual) data from 24 subjects performing various types of cheating while taking online exams. Extensive experimental results demonstrate the accuracy, robustness, and efficiency of our online exam proctoring system.

KW - Covariance feature

KW - gaze estimation

KW - online exam proctoring (OEP)

KW - phone detection

KW - speech detection

KW - text detection

KW - user verification

UR - http://www.scopus.com/inward/record.url?scp=85021323599&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85021323599&partnerID=8YFLogxK

U2 - 10.1109/TMM.2017.2656064

DO - 10.1109/TMM.2017.2656064

M3 - Article

VL - 19

SP - 1609

EP - 1624

JO - IEEE Transactions on Multimedia

T2 - IEEE Transactions on Multimedia

JF - IEEE Transactions on Multimedia

SN - 1520-9210

IS - 7

M1 - 7828141

ER -