Automated Online Exam Proctoring

Yousef Atoum, Liping Chen, Alex X. Liu, Stephen D.H. Hsu, Xiaoming Liu

    Research output: Research - peer-reviewArticle

    Abstract

    Massive open online courses and other forms of remote education continue to increase in popularity and reach. The ability to efficiently proctor remote online examinations is an important limiting factor to the scalability of this next stage in education. Presently, human proctoring is the most common approach of evaluation, by either requiring the test taker to visit an examination center, or by monitoring them visually and acoustically during exams via a webcam. However, such methods are labor intensive and costly. In this paper, we present a multimedia analytics system that performs automatic online exam proctoring. The system hardware includes one webcam, one wearcam, and a microphone for the purpose of monitoring the visual and acoustic environment of the testing location. The system includes six basic components that continuously estimate the key behavior cues: user verification, text detection, voice detection, active window detection, gaze estimation, and phone detection. By combining the continuous estimation components, and applying a temporal sliding window, we design higher level features to classify whether the test taker is cheating at any moment during the exam. To evaluate our proposed system, we collect multimedia (audio and visual) data from 24 subjects performing various types of cheating while taking online exams. Extensive experimental results demonstrate the accuracy, robustness, and efficiency of our online exam proctoring system.

    LanguageEnglish (US)
    Article number7828141
    Pages1609-1624
    Number of pages16
    JournalIEEE Transactions on Multimedia
    Volume19
    Issue number7
    DOIs
    StatePublished - Jul 1 2017

    Profile

    Education
    Monitoring
    Microphones
    Scalability
    Acoustics
    Personnel
    Hardware
    Testing

    Keywords

    • Covariance feature
    • gaze estimation
    • online exam proctoring (OEP)
    • phone detection
    • speech detection
    • text detection
    • user verification

    ASJC Scopus subject areas

    • Signal Processing
    • Media Technology
    • Computer Science Applications
    • Electrical and Electronic Engineering

    Cite this

    Automated Online Exam Proctoring. / Atoum, Yousef; Chen, Liping; Liu, Alex X.; Hsu, Stephen D.H.; Liu, Xiaoming.

    In: IEEE Transactions on Multimedia, Vol. 19, No. 7, 7828141, 01.07.2017, p. 1609-1624.

    Research output: Research - peer-reviewArticle

    Atoum Y, Chen L, Liu AX, Hsu SDH, Liu X. Automated Online Exam Proctoring. IEEE Transactions on Multimedia. 2017 Jul 1;19(7):1609-1624. 7828141. Available from, DOI: 10.1109/TMM.2017.2656064
    Atoum, Yousef ; Chen, Liping ; Liu, Alex X. ; Hsu, Stephen D.H. ; Liu, Xiaoming. / Automated Online Exam Proctoring. In: IEEE Transactions on Multimedia. 2017 ; Vol. 19, No. 7. pp. 1609-1624
    @article{159cadc63c2b47fcb11b54373b6e6e2c,
    title = "Automated Online Exam Proctoring",
    abstract = "Massive open online courses and other forms of remote education continue to increase in popularity and reach. The ability to efficiently proctor remote online examinations is an important limiting factor to the scalability of this next stage in education. Presently, human proctoring is the most common approach of evaluation, by either requiring the test taker to visit an examination center, or by monitoring them visually and acoustically during exams via a webcam. However, such methods are labor intensive and costly. In this paper, we present a multimedia analytics system that performs automatic online exam proctoring. The system hardware includes one webcam, one wearcam, and a microphone for the purpose of monitoring the visual and acoustic environment of the testing location. The system includes six basic components that continuously estimate the key behavior cues: user verification, text detection, voice detection, active window detection, gaze estimation, and phone detection. By combining the continuous estimation components, and applying a temporal sliding window, we design higher level features to classify whether the test taker is cheating at any moment during the exam. To evaluate our proposed system, we collect multimedia (audio and visual) data from 24 subjects performing various types of cheating while taking online exams. Extensive experimental results demonstrate the accuracy, robustness, and efficiency of our online exam proctoring system.",
    keywords = "Covariance feature, gaze estimation, online exam proctoring (OEP), phone detection, speech detection, text detection, user verification",
    author = "Yousef Atoum and Liping Chen and Liu, {Alex X.} and Hsu, {Stephen D.H.} and Xiaoming Liu",
    year = "2017",
    month = "7",
    doi = "10.1109/TMM.2017.2656064",
    volume = "19",
    pages = "1609--1624",
    journal = "IEEE Transactions on Multimedia",
    issn = "1520-9210",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",
    number = "7",

    }

    TY - JOUR

    T1 - Automated Online Exam Proctoring

    AU - Atoum,Yousef

    AU - Chen,Liping

    AU - Liu,Alex X.

    AU - Hsu,Stephen D.H.

    AU - Liu,Xiaoming

    PY - 2017/7/1

    Y1 - 2017/7/1

    N2 - Massive open online courses and other forms of remote education continue to increase in popularity and reach. The ability to efficiently proctor remote online examinations is an important limiting factor to the scalability of this next stage in education. Presently, human proctoring is the most common approach of evaluation, by either requiring the test taker to visit an examination center, or by monitoring them visually and acoustically during exams via a webcam. However, such methods are labor intensive and costly. In this paper, we present a multimedia analytics system that performs automatic online exam proctoring. The system hardware includes one webcam, one wearcam, and a microphone for the purpose of monitoring the visual and acoustic environment of the testing location. The system includes six basic components that continuously estimate the key behavior cues: user verification, text detection, voice detection, active window detection, gaze estimation, and phone detection. By combining the continuous estimation components, and applying a temporal sliding window, we design higher level features to classify whether the test taker is cheating at any moment during the exam. To evaluate our proposed system, we collect multimedia (audio and visual) data from 24 subjects performing various types of cheating while taking online exams. Extensive experimental results demonstrate the accuracy, robustness, and efficiency of our online exam proctoring system.

    AB - Massive open online courses and other forms of remote education continue to increase in popularity and reach. The ability to efficiently proctor remote online examinations is an important limiting factor to the scalability of this next stage in education. Presently, human proctoring is the most common approach of evaluation, by either requiring the test taker to visit an examination center, or by monitoring them visually and acoustically during exams via a webcam. However, such methods are labor intensive and costly. In this paper, we present a multimedia analytics system that performs automatic online exam proctoring. The system hardware includes one webcam, one wearcam, and a microphone for the purpose of monitoring the visual and acoustic environment of the testing location. The system includes six basic components that continuously estimate the key behavior cues: user verification, text detection, voice detection, active window detection, gaze estimation, and phone detection. By combining the continuous estimation components, and applying a temporal sliding window, we design higher level features to classify whether the test taker is cheating at any moment during the exam. To evaluate our proposed system, we collect multimedia (audio and visual) data from 24 subjects performing various types of cheating while taking online exams. Extensive experimental results demonstrate the accuracy, robustness, and efficiency of our online exam proctoring system.

    KW - Covariance feature

    KW - gaze estimation

    KW - online exam proctoring (OEP)

    KW - phone detection

    KW - speech detection

    KW - text detection

    KW - user verification

    UR - http://www.scopus.com/inward/record.url?scp=85021323599&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85021323599&partnerID=8YFLogxK

    U2 - 10.1109/TMM.2017.2656064

    DO - 10.1109/TMM.2017.2656064

    M3 - Article

    VL - 19

    SP - 1609

    EP - 1624

    JO - IEEE Transactions on Multimedia

    T2 - IEEE Transactions on Multimedia

    JF - IEEE Transactions on Multimedia

    SN - 1520-9210

    IS - 7

    M1 - 7828141

    ER -