The effectiveness of a 5-unit college-level Japanese language course taught with online supplemental material was examined against the same course taught without online material. Participants were 86 students in 8 sections of a first-semester elementary Japanese language course during a 4-year period. Complete data (quiz, midterm, and final exam scores) were available for 39 students in the comparison group and 47 students in the experimental group. The instructor used a conventional textbook and a Kana workbook based on an eclectic approach. The results showed no statistically significant differences on the students' test scores between the comparison group and the experimental group on 9 quizzes and final but showed statistically significant differences on one quiz and midterm (without-online students scored higher than with-online students). Dropout rates were significantly lower in the with-online course.