Generic Skills Measurement of Students at Fukuoka Dental College: The Usefulness of the Progress Report on Generic Skills (PROG) Test | Open Access Journals

e-ISSN: 2320-7949 and p-ISSN: 2322-0090

Generic Skills Measurement of Students at Fukuoka Dental College: The Usefulness of the Progress Report on Generic Skills (PROG) Test

Uchida Ryuji1*, Kodama Jun2, Maruta Michito3, Okamoto Fujio4, Kawaguchi Tomohiro5, Ohgi Kimiko6 and Ishikawa Hiroyuki7

1Educational Support and Institutional Research Office, Fukuoka Dental College, Fukuoka, Japan

2Department of Morphological Biology, Fukuoka Dental College, Fukuoka, Japan

3Department of Dental Engineering, Fukuoka Dental College, Fukuoka, Japan

4Department of Physiological Science and Molecular Biology, Fukuoka Dental College, Fukuoka, Japan

5Department of Oral Rehabilitation, Fukuoka Dental College, Fukuoka, Japan

6Department of Odontology, Fukuoka Dental College, Fukuoka, Japan

7Fukuoka Dental College, Fukuoka, Japan

*Corresponding Author:
Uchida Ryuji
Educational Support and Institutional Research Office
Fukuoka Dental College 2-15-1 Tamura
Sawara-ku, Fukuoka, 814-0193, Japan
Tel: +81-92- 801-0411
Fax: +81-92-801-0427
E-mail: ryu@college.fdcnet.ac.jp

Received Date: 18/01/2018; Accepted Date: 23/01/2018; Published Date: 30/01/2018

Visit for more related articles at Research & Reviews: Journal of Dental Sciences

Abstract

In recent years, university education has been called upon to cultivate new abilities known as “generic skills,” which have been defined in various ways by the different ministries and agencies advocating their necessity. These generic skills are also indispensable for dentistry graduates and skills training is important in the context of dental education. However, while conventional dental education has heretofore conducted assessments with regard to knowledge, understanding and professional ability when evaluating learning results, higherorder integrative capabilities such as generic skills have not been evaluated. Therefore, after introducing the Progress Report on Generic Skills (PROG) Test as an evaluation method for measuring generic skills to examine their relationship with academic achievement. Administering PROG Test soon after the time of admission, it was possible to measure generic skills at the time of admission, within which the overall literacy results were related to academic ability at the time of admission and subsequent academic performance. We can report the suggested possibility that the PROG test may be used to evaluate generic skills among dentistry students.

Keywords

Generic skills, PROG test, Evaluation, Dentistry students

Introduction

In recent years, university education has been called upon to cultivate new abilities known as “generic skills”. Generic skills, also known as “translatable skills,” are high-order skills that can be applied in a variety of circumstances, such as creativity, flexibility, independence, ability to work in a team, communicative ability, critical thinking, time management, leadership, planning ability, and self-management [1]. These skills have been variously defined by the different Japanese government ministries and agencies advocating their necessity, as with the term “human power” proposed by the Cabinet Office [2], “core skills for working adults” proposed by the Ministry of Economy, Trade and Industry (METI) [3], “academic competency” proposed by the Central Education Council [4], “employability” proposed by the Ministry of Education, Culture, Sports, Science and Technology (MEXT) [5], and “fundamental general-purpose skills” in the field of career education [6].

Dental education requires not only the interpretive and thinking abilities that accompany the simple memories gradually cultivated from early education through high school but also the general-purpose skills such as clinical reasoning and the integration of learning content from individual subjects taken thus far; these skills are also indispensable abilities for dentists to have upon graduation. Although Kawashima describes generic skills training by such means as Project-Based Learning internships and service learning [1], METI favors skills training in programs related to university specializations over special programs aiming solely at generic skills training [7]. However, very few universities carry out generic skills training over the entire undergraduate curriculum, with most undertaking such training in the context of first-year introductory and liberal arts subjects [8]. At Fukuoka Dental College, we have introduced an initiative that aims to carry out generic skills training in our first-year introductory and liberal arts subjects as well as in the entire six-year undergraduate curriculum, and then evaluate and visualize the skills that have been acquired. However, while conventional dental education has heretofore conducted assessments with regard to knowledge, understanding, and professional ability when evaluating learning results, higher-order integrative capabilities such as generic skills have not been evaluated.

Therefore, in the academic year 2016, we introduced the Progress Report on Generic Skills (PROG) Test to measure generic skills, and have examined the actual extent and nature of the relationship between generic skills at the time of admission, academic ability at the time of admission and Grade Point Average (GPA) in Year 1.

Target and Methods

Target

In the beginning, the survey was explained orally and via a document, and after checking the consent, it was conducted.

1. When it is used for any purpose other than study support, it becomes anonymous to connect all data and process and use it so that an individual cannot specify.

2. You can refuse use of data.

3. Opt-out is possible, even if it once agrees.

Get consent for use of education data from students and parents in research, upon admission than 2018. The period before it, after explaining the above mentioned contents to a student, was put into effect. Although all the target students' consent was checked orally, a less than 20-year-old guardian's consent is not checking this time. Thus, this research was done by having adopted the principle of Declaration of Helsinki, and was performed by obtaining the approval of Ethical Review Board, Fukuoka Dental College.

The study involved 97 students, all of whom were students enrolled in Year 1 in the academic year 2016, and who appeared for the Comprehensive Academic Achievement Examination (CAAE) and PROG Test conducted on April 12, 2016. The competency elements of the PROG Test results for 94 of these students were used, after 3 of the 97 students were identified as ineligible due to an insufficient number of completed responses.

Analysis of PROG Test Results

Tables 1 and 2 shows an outline of the PROG Test. For the literacy test, we used a 7-stage judgment result for overall literacy, and for the 6 items comprising 4 problem-solving skills and 2 processing skills, we used a 5-stage judgment result. For the competency test, we used a 7-stage judgment result for the overall competency and 3 competency skills, and a 5-stage judgment result for the 9 constituent elements comprising these 3 competencies.

Table 1. PROG (progress report on generic skills) test outline.

Literacy Test Literacy
Format Mark format   6 skills  
Questions 30 problems Overall Literacy Problem-solving skills Data collection
Data analysis
Identifying Problems
Conceptual Ability
Expressiveness
Execution
 4 Skills
Not measured
Time allowed 45 minutes
Areas measured 1. Problem-solving ability (Data Collection/Data Analysis/Identifying Problems/Conceptual Ability)
2. "Verbal" and "Non-verbal“ Processing Skills
Processing Skills Verbal  
Non-verbal  
Competency Test Competency
Format Mark format    3 skills 9 Elements  
Questions Two-Sided Selection Method      195 questions Overall
Competency
Basic Problem-Oriented Skills Identifying Problems
Planning
Practical Ability
 
Hypothetical Scenario (short answer) 50 questions Basic Interpersonal Skills Affinity
Cooperation
Leadership
 
Hypothetical Scenario (long answer) 6 questions Basic Self-Management Skills Emotional Control
Creative Confidence
Perseverance
 
 Total 251 questions
 Time allowed  40 minutes   
Areas measured Basic Problem-Oriented Skills (Identifying Problems/Planning/Practical Ability)
Basic Interpersonal Skills (Affinity/Cooperation/Leadership)
Basic Self-Management Skills (Emotional Control/Creative Confidence/Perseverance)

Table 2. Evaluation scores and level settings for the PROG Test.

Literacy
  1 2 3 4 5 6 7
 Overall Evaluation Level at which the acquisition of the basic literacy expected of a university student is required. Level at which the basic literacy expected of a university student has been acquired to some degree. Level at which the basic literacy expected of a university student can be acquired with a little more effort. Level at which the basic literacy expected of a university student has been acquired (first-year level) Level at which the basic literacy expected of a member of the workforce has been acquired to some degree. Level at which the basic literacy expected of a member of the workforce can be acquired with a little more effort. Level at which the basic literacy expected of a member of the workforce has been acquired.
4 Skills      Processing Skills Has the basic ability to solve simple problems. Understands basic problem-solving skills to some degree. Understands basic problem-solving skills. Understands problem-solving skills, and is able to solve problems. Understands problem-solving skills, and is able to solve problems with contextual complexity.    
Competency
  1 2 3 4 5 6 7
 Overall Evaluation       3 Skills Level at which a student is not now able to exercise an ability, but is very likely to grow significantly with the right impetus. Level at which a student can achieve things in his or her own way, but does not meet societal expectations. Level that meets societal expectations to some degree. Level that meets societal expectations. Level at which the student can exceed societal expectations as an individual. Level at which the student inspires the team as a leader, earning societal approval. Innovative level at which the student can inspire those around him or her to change their circumstances.
Constituent Elements of Each Competency Level at which a student is not now able to exercise an ability, but is very likely to grow significantly with the right impetus. Level at which the student can meet societal expectations with a little more effort. Level that meets societal expectations. Level at which the student can exceed societal expectations. Innovative level at which the student can inspire those around him or her to change their circumstances.    

Data Used

We used the total scores for the PROG Test and the CAAE administered on the same day, students’ Year 1 GPA for 2016 and their grades for 22 subjects offered in Year 1. After linking this data using the PROG Test results and student identification numbers, the student identification numbers were deleted so that the data could be used as unlinkable anonymized data.

Statistical Processing

To investigate the relationships among the PROG Test results, academic ability at the time of admission, and learning results after admission, we calculated Spearman’s rank correlation coefficient with respect to the judgment result of each element of the PROG Test and the overall CAAE scores, Year 1 GPA scores, and grades for 22 subjects offered in Year 1.

Results

The averages of the results for overall CAAE scores, Year 1 GPA, grades for subjects offered in Year 1, and PROG Test measurement items are shown in Tables 3-5.

Table 3. Total comprehensive academic achievement exam (CAAE) scores and mean GPA for year 1.

   Total CAAE Scores Year 1 GPA
Mean 287.1 3.1
SD 61.84 0.53

Table 4. Mean scores for subjects offered in year 1.

  Basic Mathematics General Mathematics Basic Science/ Physics Basic Physics Basic Science/ Chemistry Basic Chemistry Basic Science/ Biology Cell Biology Cell Chemistry Medical Engineering Introduction to Anatomy
Mean 82.4 79.2 81.3 67.2 82.9 71.8 82.3 73.7 66.3 79.8 65.1
SD 14.17 12.07 11.08 12.81 15.18 13.62 11.96 12.68 13.5 14.61 12.97
  Scientific English I Practical English I Scientific English II Practical English II Modern Civilization Long-Term Care Facility/Brushing Experience Introduction to Medicine/Oral Medicine Basic Operation Exercise Data Processing Exercise I Data Processing Exercise II Debate Exercise
Mean 81.5 75.6 72.4 76.5 91.5 85.2 80.8 95.2 98.4 97.1 72
SD 13.25 11.78 14.82 11.49 9.65 5.01 7.08 5.77 4.55 7.08 6.17

Table 5. Mean scores for PROG test measurement items.

Literacy
    4 Skills Processing Skills
   Totals Data Collection Data Analysis Identifying Problems Conceptual Ability Verbal  Non-Verbal
Mean 5.1 3.4 3.7 4.1 4.3 3.6  4.4
SD 1.37 1.33 1.22 1.11 0.85 1.12  0.75
Competency 9 Elements
    3 Skills Basic Interpersonal Skills Basic Self-Management Skills Basic Problem-Oriented Skills
   Totals Basic Interpersonal Skills Basic Self-Management Skills Basic Problem-Oriented Skills Affinity Cooperation Leadership Emotional Control Creative Confidence Perseverance Identifying Problems Planning Practical Ability
Mean 3.3 3.7 3.3 3.7 4.3 3.8 3.1 3.2 3.3 3.5 3.8 3.7 3.6
SD 1.6 1.72 1.56 1.59 1.85 1.78 1.7 1.62 1.7 1.62 1.76 1.73 1.43

Correlations among overall CAAE Scores, Year 1 GPA, and PROG Test Measurement Items (Table 6).

Table 6. Correlations among total CAAE scores, year 1 GPA and PROG test measurement items.

   Total CAAE Scores Year 1 GPA
Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance
Literacy Totals 0.303** 0.003 0.351** 0
Competency Totals -0.145 0.163 -0.041 0.695
**Correlation coefficient is significant at the 1% level (both sides)
*Correlation coefficient is significant at the 5% level (both sides)
Literacy  Total CAAE Scores Year 1 GPA
Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance
Data Collection 0.199 0.05 0.187 0.067
Data Analysis 0.300** 0.003 0.247* 0.015
Identifying Problems 0.245* 0.016 0.400** 0
Conceptual Ability 0.098 0.341 0.191 0.061
Verbal Processing Skills 0.202* 0.047 0.112 0.277
Non-verbal Processing Skills 0.147 0.151 0.062 0.545
**Correlation coefficient is significant at the 1% level (both sides)
 *Correlation coefficient is significant at the 5% level (both sides)
Competency  Total CAAE Scores Year 1 GPA
Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance
Basic Interpersonal Skills -0.197 0.058 -0.045 0.664
Basic Self-Management Skills 0.03 0.776 0.087 0.405
Basic Problem-Oriented Skills -0.034 0.744 -0.041 0.693
** Correlation coefficient is significant at the 1% level (both sides)
* Correlation coefficient is significant at the 5% level (both sides)

Relationships between overall CAAE scores and PROG test measurement items: While a significant positive correlation was found between overall CAAE scores and overall literacy (ρs=0.303, p<0.01), no significant correlation was found with overall competency. In terms of literacy elements, significant positive correlations were found with the ability to analyze data, the ability to identify problems, and verbal processing ability (data analysis: ρs=0.300, p<0.01, ability to identify problems: ρs=0.245, p<0.05, verbal processing ability: ρs=0.202, p<0.05). In terms of competency elements, no significant correlations were found with overall CAAE scores for any items.

Correlations between scores for subjects offered in year 1 and PROG test evaluation items (Tables 7-9).

Table 7. Correlations between scores for subjects offered in year 1 and PROG test evaluation items.

   Total CAAE scores Year 1 GPA
Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance
Literacy Totals 0.303** 0.003 0.351** 0
Competency Totals -0.145 0.163 -0.041 0.695
** Correlation coefficient is significant at the 1% level (both sides)
* Correlation coefficient is significant at the 5% level (both sides)
Literacy  Total CAAE scores Year 1 GPA
Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance
Data collection 0.199 0.05 0.187 0.067
Data analysis 0.300** 0.003 0.247* 0.015
Identifying Problems 0.245* 0.016 0.400** 0
Conceptual Ability 0.098 0.341 0.191 0.061
Verbal Processing Skills 0.202* 0.047 0.112 0.277
Non-verbal Processing Skills 0.147 0.151 0.062 0.545
** Correlation coefficient is significant at the 1% level (both sides)
* Correlation coefficient is significant at the 5% level (both sides)
Competency  Total CAAE Scores Year 1 GPA
Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance
Basic Interpersonal Skills -0.197 0.058 -0.045 0.664
Basic Self-Management Skills 0.03 0.776 0.087 0.405
Basic Problem-Oriented Skills -0.034 0.744 -0.041 0.693
** Correlation coefficient is significant at the 1% level (both sides)
* Correlation coefficient is significant at the 5% level (both sides)

Table 8. Correlations between scores for subjects offered in year 1 and PROG test evaluation items.

  Cell Chemistry Medical Engineering Introduction to Anatomy Scientific English I Practical English I Scientific English II Practical English II Modern Civilization
Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance
Literacy Totals 0.349** 0 0.328** 0.001 0.237* 0.02 0.311** 0.002 0.261* 0.01 0.306** 0.002 0.430** 0 0.281** 0.006
Competency Totals 0.029 0.782 -0.001 0.992 0.012 0.907 -0.061 0.562 0.082 0.434 -0.065 0.534 0.135 0.196 0.086 0.413
** Correlation coefficient is significant at the 1% level (both sides)
* Correlation coefficient is significant at the 5% level (both sides) 
Literacy Cell Chemistry Medical Engineering Introduction to Anatomy Scientific English I Practical English I Scientific English II Practical English II Modern Civilization
Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance
Data Collection 0.195 0.057 0.208* 0.042 0.136 0.187 0.205* 0.045 0.053 0.61 0.260* 0.01 0.274** 0.007 0.166 0.106
Data Analysis 0.14 0.174 0.277** 0.006 0.045 0.665 0.129 0.211 0.197 0.054 0.126 0.222 0.334** 0.001 0.167 0.103
Identifying Problems 0.486** 0 0.356** 0 0.286** 0.005 0.302** 0.003 0.227* 0.026 0.327** 0.001 0.283** 0.005 0.211* 0.039
Conceptual Ability 0.192 0.062 0.216* 0.035 0.086 0.407 0.158 0.124 0.233* 0.023 0.138 0.181 0.369** 0 0.283** 0.005
Verbal Processing Skills 0.065 0.527 0.241* 0.018 -0.021 0.836 0.075 0.467 0.118 0.254 0.02 0.845 0.199 0.052 0.109 0.292
Non-Verbal Processing Skills -0.055 0.597 0.126 0.222 -0.004 0.972 0.086 0.402 0.176 0.087 0.147 0.153 0.172 0.093 0.03 0.77
** Correlation coefficient is significant at the 1% level (both sides)
* Correlation coefficient is significant at the 5% level (both sides)
Competency Cell Chemistry Medical Engineering Introduction to Anatomy Scientific English I Practical English I Scientific English II Practical English II Modern Civilization
Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance
Basic Interpersonal Skills 0.004 0.971 -0.013 0.905 0.015 0.888 -0.109 0.299 0.038 0.715 -0.136 0.192 0.084 0.426 0.022 0.831
Basic Self-Management Skills 0.053 0.616 0.114 0.276 0.157 0.132 0.11 0.292 0.058 0.578 0.061 0.558 0.222* 0.032 0.167 0.11
Basic Problem-Oriented Skills -0.004 0.973 -0.176 0.091 -0.029 0.779 0.041 0.696 -0.009 0.932 -0.066 0.532 0.007 0.945 0.168 0.106
** Correlation coefficient is significant at the 1% level (both sides)
* Correlation coefficient is significant at the 5% level (both sides)

Table 9. Correlations between scores for subjects offered in year 1 and PROG test evaluation items.

  Long-Term Care Facility/Brushing Experience Introduction to Medicine/Oral Medicine Basic Operation Exercise Data Processing Exercise I Data Processing Exercise II Debate Exercise
Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance
Literacy Totals 0.186 0.069 0.354** 0 0.335** 0.001 0.02 0.85 0.184 0.073 0.262** 0.01
Competency Totals 0.119 0.257 0.081 0.44 0.061 0.561 0.081 0.442 0.048 0.65 0.257* 0.013
** Correlation coefficient is significant at the 1% level (both sides)
* Correlation coefficient is significant at the 5% level (both sides)
Literacy Long-Term Care Facility/Brushing Experience Introduction to Medicine/Oral Medicine Basic Operation Exercise Data Processing Exercise I Data Processing Exercise II Debate Exercise
Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance
Data Collection 0.339** 0.001 0.14 0.174 0.205* 0.045 -0.055 0.597 0.199 0.052 0.077 0.456
Data Analysis 0.122 0.236 0.307** 0.002 0.266** 0.009 0.076 0.461 0.156 0.13 0.210* 0.04
Identifying Problems 0.07 0.497 0.204* 0.046 0.205* 0.045 0.061 0.556 0.061 0.555 0.18 0.08
Conceptual Ability 0.222* 0.029 0.208* 0.042 0.268** 0.008 -0.039 0.704 0.095 0.359 0.164 0.11
Verbal Processing Skills 0.102 0.323 0.156 0.129 0.221* 0.031 0.097 0.348 0.186 0.069 0.021 0.841
Non-Verbal Processing Skills 0.014 0.891 0.156 0.13 0.176 0.087 -0.04 0.702 0.206* 0.044 0.019 0.851
** Correlation coefficient is significant at the 1% level (both sides)
* Correlation coefficient is significant at the 5% level (both sides)
Competency Long-Term Care Facility/Brushing Experience Introduction to Medicine/Oral Medicine Basic Operation Exercise Data Processing Exercise I Data Processing Exercise II Debate Exercise
Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance
Basic Interpersonal Skills 0.113 0.283 0.085 0.42 0.041 0.697 0.064 0.544 0.065 0.538 0.214* 0.039
Basic Self-Management Skills 0.174 0.096 0.119 0.255 0.051 0.629 0.025 0.811 -0.014 0.893 0.286** 0.005
Basic Problem-Oriented Skills -0.139 0.185 0.078 0.455 0.065 0.536 -0.009 0.935 -0.098 0.35 0.113 0.281
** Correlation coefficient is significant at the 1% level (both sides)
* Correlation coefficient is significant at the 5% level (both sides)

Relationships between grades in subjects offered in year 1 and overall literacy/overall competency for PROG test measurement items: Among the 22 subjects offered in Year 1, while all but 3 subjects (“Long-Term Care Facility/Brushing Experience,” “Data Processing Exercises,” “Data Processing Exercises II”) were found to have a significant positive correlation with overall literacy, only the “Debate Exercise” course was found to have a significant positive correlation with overall competency (ρs=0.257, p<0.05).

Relationships between grades in subjects offered in year 1 and PROG test measurement items literacy elements (4 Problem- Solving Skills and 2 Processing Skills): Among the 22 subjects offered in Year 1, significant positive correlations were found with the literacy element of data collection in 10 subjects, of data analysis in 10 subjects, of the ability to identify problems in 17 subjects, of conceptual ability in 9 subjects, of verbal processing ability in 4 subjects, and of non-verbal processing ability in 2 subjects. Looking at each course in turn, significant positive correlations were found with 5 elements in 2 subjects, 4 elements in 3 subjects, 3 elements in 3 subjects, 2 elements in 8 subjects, and 1 element in 5 subjects, with only 1 course in which no significant correlations were observed with any element.

Relationships between grades in subjects offered in year 1 and PROG test measurement items competency elements (3 Skills): Among the 22 subjects offered in Year 1, correlations were found with the 3 subjects of “Basic Science and Physics,” “Practical English II,” and “Debate Exercise”; significant positive correlations in Basic Science and Physics were observed with regard to the two skill sets of Basic Self-Management Skills (ρs=0.284, p<0.05) and Basic Problem-Oriented Skills (ρs=0.342, p<0.01), in Practical English II with regard to the skill set of Basic Self-Management Skills (ρs=0.222, p<0.05), and in Debate Exercise with regard to the two skill sets of Basic Interpersonal Skills (ρs=0.214, p<0.05) and Basic Self-Management Skills (ρs=0.286, p<0.01). The relationship between grades for these 3 subjects and the 9 constituent competency elements of these 3 skill sets is shown in Table 10. Significant positive correlations in Basic Science and Physics were observed with the two elements of emotional control (ρs=0.361, p<0.01) and the ability to identify problems (ρs=0.381, p<0.01), in Practical English II with the three elements of Leadership (ρs=0.207, p<0.05), creative confidence (ρs=0.244, p<0.05), and perseverance (ρs=0.215, p<0.05) and in Debate Exercise with the four elements of leadership (ρs=0.270, p<0.01), creative confidence (ρs=0.305, p<0.01), ability to identify problems (ρs=0.215, p<0.05), and practical ability (ρs=0.256, p<0.05).

Table 10. Correlations between competency elements (9 Elements) and scores for 3 subjects found to have a significant positive correlation with competency elements (3 Skills).

Competency Basic Science/Physics Practical English II Debate Exercise
Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance Correlation Coefficient Probability of Significance
Basic Interpersonal Skills Affinity 0.04 0.763 0.066 0.533 0.132 0.21
Cooperation 0.055 0.676 -0.018 0.864 0.155 0.14
Leadership 0.066 0.614 0.207* 0.048 0.270** 0.009
Basic Self-Management Skills Emotional Control 0.361** 0.005 0.144 0.171 0.202 0.053
Creative Confidence 0.222 0.088 0.244* 0.019 0.305** 0.003
Perseverance 0.145 0.268 0.215* 0.039 0.182 0.082
Basic Problem-Oriented Skills Identifying Problems 0.381** 0.003 0.071 0.5 0.215* 0.039
Planning 0.193 0.14 -0.072 0.495 -0.002 0.988
Practical Ability 0.193 0.139 0.099 0.347 0.256* 0.014
** Correlation coefficient is significant at the 1% level (both sides)
* Correlation coefficient is significant at the 5% level (both sides)

Discussion and Conclusion

The PROG report used in this investigation is an assessment program, jointly developed by the Kawaijuku Educational Institution and RIASEC Inc., that supports the growth of general-purpose proficiencies, attitudes, and inclinations (generic skills) required by society. The PROG Test is an assessment test that objectively measures the skills required by society in terms of the two aspects of literacy and competency [7-10]. It has been used in various universities to measure learning outcomes with a view of educational improvement, as well as of encouraging student growth and subjective learning. Since its release in April, 2012, the PROG Test has already been administered at approximately 290 universities across Japan and taken by more than a quarter of a million students [11]. With the PROG Test, it is possible to objectively grasp the current status of individual students with measurements from the two standpoints of literacy and competency.

Using the items shown in Table 1, with regard to literacy, we check the skills to solve challenges by using knowledge of new problems and problems that have not previously been experienced, and with regard to competency, the behavioral styles and decision-making characteristics acquired to better cope with contextual circumstances [9,10]. Evaluation scores and level setting are shown in Table 2. For the evaluation of overall literacy, Stages 1 to 4 are set at the level achieved in the first year of study, with Stage 7 set at the level of someone who has completed an undergraduate program. For the evaluation of overall competency, Stages 1 to 4 are set at the level of someone who has completed an undergraduate program, while Stages 5 and 6 are set at the level expected for graduates who have been in the workforce for up to three years, and Stage 7 is set at the level of someone playing a leading role as a young worker [11].

Narita has pointed out that while a clear positive correlation is apparent in the relationship between PROG Test results and the Standard Deviation (SD) value for entrance examinations, in the sense that higher entrance examination SD values are associated with higher literacy scores, the relationship with regard to competency does not appear to be as strong as that with literacy; in terms of overall competency, very little difference was found among the High-SD Group (SD ≥ 55), Medium-SD Group (SD ≥ 45<55), and Low-SD Group (SD value< 45) [12]. Also, Yamamoto and Matsumoto state that basic academic ability up to high school is a necessary condition for literacy [13]. In the PROG Test results for first-year Fukuoka Dental College students in 2016 as well, while a significant positive correlation with overall literacy was found with overall CAAE scores, which measure basic academic ability at the time of admission, no significant correlation was found with respect to overall competency. Looking at the specific literacy elements with which correlations were found, we find that significant positive correlations were observed with respect to data analysis, the ability to identify problems, and verbal processing ability.

Yamamoto discussed the relationship between PROG Test results and GPA scores, reporting that test results do not necessarily correlate with GPA [14,15]. However, we found a correlation with GPA in the PROG Test results for first-year Fukuoka Dental College students in 2016. In the measurement items, while a significant positive correlation was found with respect to overall literacy, no significant correlation was found with respect to overall competency. Looking at the specific literacy elements with which correlations were found, we find that significant positive correlations were observed with respect to data analysis and the ability to identify problems. In terms of the correlation with GPA, given that the subjects offered in Year 1 at our school promote the development of basic academic ability and include several remedial education subjects in mathematics and science for advancing students to the next stage, it seems possible that this correlation may be due to the fact that basic academic ability at the time of admission is reflected in Year 1 GPA scores.

This study identified that by administering the PROG Test soon after the time of admission, it was possible to measure generic skills at the time of admission, within which the overall literacy results were related to academic ability at the time of admission and the Year 1 GPA scores after the first year of study. In addition, there is a possibility that this may also be related to subsequent academic performance, suggesting its usefulness for the instruction of individual students. In terms of the competency aspect of generic skills, no significant correlations were observed among PROG Test results, CAAE scores, and Year 1 GPA. However, the ability to respond well to contextual circumstances by taking advantage of the behavioural styles and decision-making characteristics acquired measured from this competency standpoint also represent an indispensable capability for engaging in dentistry. Yamamoto states that while university education does foster competency, its evaluation has been lacking such that the assessment of required subjects and their results while being cognizant of generic skills training remains a challenge [13]. Among the subjects offered in Year 1 at our school, there were some subjects in which a significant positive correlation was found between grade scores and competency-related measurement items. These subjects included observational evaluation such as interpersonal dialog and group work performance, and while this seems to be due to consistency between evaluation items and competencyrelated measurement items, this needs to be studied further in the future. Also, as reported by Sasakawa [16,17] and Saga et al. [18]. It appeared that it would also be possible to evaluate the growth of students’ generic skills by administering the PROG Test again after a period of several years and comparing the results.

Acknowledgments

This study was conducted with the support of the MEXT Acceleration Program for University Education Rebuilding. It should also be noted in relation to this presentation that there are no company-related conflicts of interest to disclose.

References