PBM E-Questionnaire: Application Development and UI / UX Analysis

Received 2020-07-28 Revised 2020-12-02 Accepted 2020-12-07 Batam State Polytechnic declares itself as a tertiary institution which is very concerned about the quality of its education. This achievement was reported through the implementation of an education quality assurance system through study programs accredited by BAN-PT. One of the quality assurance services for education is through feedback studies on learning services through study programs. Almost every year in Batam State Polytechnic, study programs apply for accreditation to BAN-PT. Then along with these activities needed data quickly and completely. However, in reality, it is not uncommon for the drafting team to experience obstacles in obtaining the data. Based on this, in this study, an electronic PBM questionnaire application was made from several parties including lecturers, students, alumni, and alumni users (stakeholders). Making an application using the Research and Development (RnD) method and analysis in the Interface and Experience section. The results of the study with the average achievement percentage reached 88.3% showed that the user interface of the E-Questionnaire website application was very well received by the user. Keyword:


I. INTRODUCTION
Batam State Polytechnic declares itself as a tertiary institution which is very concerned about the quality of its education. This achievement was reported through the implementation of an education quality assurance system including BAN-PT accredited study programs since its foundation. ISO-based Higher Education Quality Management System and Internal Quality Education System (SPMI) through the External Quality Assurance System (SPME) evaluation system by BAN-PT [1].

Gambar 1. External Quality Assurance System
One of the education quality assurance services is through a survey of the quality of learning services through study programs. A study program is a unit that implements the teaching and learning process. In Batam State Polytechnic, the study program manager is expected to be able to evaluate PBM (Teaching and Learning Process) every semester to ensure the implementation of teaching and learning is monitored and evaluated periodically. It aims to improve the quality of PBM systematically. In Batam State Polytechnic, PBM evaluates 4 times in one semester. The components as PBM material generally consist of: (1) Regarding students (in the form of Student Warning Letter and withdrawals, and others); (2) The presence of lecturers, the achievement of the material, and student questionnaires; (3) Trusteeship Data, Internship and Final Project.
One important part that needs to be analyzed is the response of parties related to the PBM process such as lecturers (lecturers), students, alumni, and users of graduates. Based on the results of the analysis, it is hoped that important information will be obtained to improve the quality of PBM.  Table   Another interest in this questionnaire data is for the submission of study program accreditation. The required questionnaire data is the same as the PBM Batam Polytechnic report. The value points obtained from this data are 1.36.

Gambar 3. PBM Batam Polytechnic State Report Table
Almost every year, study programs apply for accreditation from BAN-PT. Then along with these activities needed data quickly and completely. However, in reality, it is not uncommon for the drafting team to experience obstacles in obtaining the data. Possible factors for the existence of these constraints are the lack of good management of data storage related to the activities that have been carried out so that when the data is needed, a search or processing is carried out which results in inadequate data obtained making it difficult for the team to carry out its work, and so on.
Based on this, in this study, an electronic PBM questionnaire application was made from several parties including lecturers, students, alumni, and alumni users (stakeholders). Web-based applications that have responsive capabilities. Generally, someone when confronted with data entry (more precisely a questionnaire), gives an unfavorable impression. This is likely due to the large number of points that must be filled, difficult to fill in, media filling constraints, busyness reasons, media, and others. Therefore, through this application, it will be equipped with features to process the data and then it can be reviewed as a learning evaluation material implemented by the study program. It is expected that through this application and also the direct application to become tools (tools) for the management of study programs at the Batam State Polytechnic to conduct periodic and systematic reviews of the questionnaire. Data needed for other purposes such as accreditation can also be easily and quickly provided.
Therefore, in this research the making of an application uses the Research and Development (RnD) method and analysis in the Interface and Experience section.

II. LITERATURE REVIEW
Research on web-based systems, especially for academic activities, has been widely carried out, one of which is making applications of web-based fieldwork information systems to assist the process of street vendors activities [16]. Next is about making a website application to manage data in the implementation of an internship at the Batam State Polytechnic [14].
Web application development methods used have many types, such as the waterfall method, while building applications generally focus on the method of testing performed. One of the testing methods used is like the BlackBox method. Other testing methods can use the Heuristic Evaluation method. This method is generally used to analyze the UI and UX of a website [4]. The following details the comparative research that has been done before in Table I. 2) Product Design, web-based application design based on the need for applications as tools to help study program managers in processing PBM feedback data. 3) Product Development, application development is based on design results. 4) Distribution of products (feasibility of ready-to-use products), it is certain that the application is ready for use and the first test is using PBM data for odd semester 2019-2020, so that it can be reviewed by the study program manager.

B. Heuristic Evaluation Method
The quality of the feasibility of a website can be known through the evaluation process, one of the evaluation processes that can be done is usability evaluation which is a solution to find out the usability problem. For the system to achieve certain objectives effectively, efficiently, and achieve user satisfaction requires usability evaluation. One of them is a Heuristic evaluation (HE). HE is a usability inspection to effectively improve a design by using a set of related heuristics. The process of HE allows a set of evaluators who independently evaluate and assess the system of each heuristic that will show usability problems [2].

C. Likert Scale
Likert Scale is a conventional and popular psychometric scale applied in conducting research surveys. Likert scale is used in measuring the behavior, perceptions, and responses of individuals and groups of people relevant to the subject of social cases [17]. Regarding the Likert Scale as the answer choices for respondents and the value of each choice can be seen in Table III. The steps to calculate the results of data processing are as follows: 1. 1. Calculate the total score. The total score is calculated using the formula (1) based on a Likert scale. = × (1) T = Total number of respondents. Pn = Choice of Likert score. 2. Calculate the percentage index. After the total score is obtained, the next thing to do is determine the lowest score using formula (2) and determine the index in percent using formula (3). = × (2) Y = Lowest score % = × (3) Index % = index in percent 3. Calculate the interval. After the index calculation is obtained, the next thing to do is to evaluate with the final value and be referred to at the interval. Determine the index interval using the formula (4).
With a total of five Likert scores, the index intervals can be calculated and produce the following intervals:

A. Implementation
This section will explain the implementation of the functional system that has been successfully carried out consisting of 6 functional.

1) Implement logins, change passwords, and forget passwords (F01)
The login process is carried out if the user has entered data in the form of a username and password on the login form and then presses the login button, then the system will read the data in the database whether the user data is appropriate or not. The process of changing the password is done if the user has logged in to the application. The user enters data in the form of a new password on the form change password then the user presses the submit button, then the system will update the user's data in the database. The process of forgetting the password is done if the user has entered data in the form of an email on the forgotten password form then the user presses the submit button, then the system will read the data in the database whether it is suitable or not. Then the user will be transferred to the new password creation page after the user fills in the new password data, the system will then update the user data in the database. In this section after the admin login to the application, then the admin goes to the user settings page and then the admin presses the add user button. Then the admin is required to fill in the data in the form of the name, username, email, password, and user level on the form added by the user then the admin presses the add user button. Then the system will store user data into a database. The user editing process is done if the admin has selected the user who wants to be changed then pressing the edit button on the user list, then the admin enters the data in the form of the name, username, email, password and user level on the user edit form then the admin presses the user edit button. Then the system will update user data into the database. The delete user process is done if the admin has selected the user who wants to be deleted then pressing the delete button, then the system will delete the user from the database.

3) Implementation Admin can manage (add, edit, delete)
questionnaire (F03) In this section after the admin has logged in to the application, then the admin enters the question set page then the admin presses the add question button. Then the admin is required to fill in the data in the form of questions, user level, and user objectives on the form added questions then the admin presses the add question button. Then the system will store the question data into the database. The question editing process is done if the admin has selected the question that you want to change, then presses the edit button on the question list, then the admin enters the form of data in the form of questions, user level, and user destination in the next question edit form the admin presses the question edit button.
Then the system will update the question data into the database. The delete question process is done if the admin has selected the question you want to delete then press the delete button, then the system will delete the question from the database.

4) Implementation Admin can manage (add, edit, delete) parameters (F04)
In this section after the admin has logged in to the application, then the admin enters the parameter management page and then the admin presses the add parameter button. Then the admin is required to fill in the data in the form of a parameter name on the form plus parameter then the admin presses the add parameter button.
Then the system will store parameter data into the database. The parameter editing process is done if the admin has selected the parameters to be changed and then presses the edit button in the parameter list, then the admin enters the data in the form of a parameter name on the parameter edit form then the admin presses the parameter edit button. Then the system will update the parameter data into the database. The delete parameter process is done if the admin has selected the parameters that you want to delete then press the delete button, then the system will delete the parameters from the database.

5) User Implementation (students, lecturers, alumni,
graduate users) can fill out questionnaires based on their respective access rights (F05) In this section after the user has logged in to the application, then the user enters the questionnaire contents page then the user enters the questionnaire data, and the next parameter the user presses the submit button. Then the system will store the questionnaire data into the database.

6) User Implementation (admin, lecturer, company) can see the questionnaire diagram (F06)
In this section after the user has logged in to the application, then the user enters the main page. Then the system reads the data in the database and converts it in a form.

B. Testing 1) Alpha Testing
This test is an application test by the application developer. This test is done in a BlackBox to test the successful implementation of functional requirements. 6 functionalities have been successfully implemented and tested.

2) Beta Testing
Testing this stage is the process of evaluating the user interface of the website that will be carried out by users using Heuristic evaluation. Testing will be given to 30 respondents, namely students, lecturers, and alumni using a Likert scale. Questionnaire data results from 30 respondents taken from 23 students, 3 lecturers, 6 alumni in detail: The score on each of the statements above will be calculated using the formula of the total score, ie the score of the number of respondents who chose multiplied by the choice of the Likert score. Following the calculation of the questionnaire from the test results in detail: 1) Statement H1.1 "There was a message when the login was not successful". The calculated Likert Score on the H1.1 statement is presented in the following table IV:

Questions
There was a message when the login was unsuccessful. with the statement "There is a message when the login is not successful".
2) Statement H1.2 "There is a message when not filling data correctly and or completely". The calculated Likert score on the H1.2 statement is presented in the following Table V:

Questions
There is a message when not filling data correctly and or completely. Based on the calculation of the Likert score in the H1.2 calculation table, the value of the percentage of user satisfaction with the appearance of the application reaches 90% which means the user strongly agrees with the statement "There is a message when not filling data correctly and or completely".
3) Statement H2.1 "The language used in the application is easy to understand". The calculated Likert score on the H2.1 statement is presented in Table VI below:

Questions
The language used in the application is easy to understand.

Questions
Given the freedom to access features to return. Based on the calculation of the Likert score in the calculation table H3.1, the value of the percentage of user satisfaction with the display application reaches 87.33% which means the user strongly agrees with the statement "Given the freedom in accessing features to return". 6) Statement H3.2 "Navigation in the application is easy to use". The results of calculating the Likert score in statement H3.2 are presented in the following Table IX:

Questions
Navigation on the application is easy to use. Based on the calculation of the Likert score in the H3.2 calculation table, the value of the percentage of user satisfaction with the application display reaches 88.67% which means the user strongly agrees with the statement "Navigation on the application is easy to use". 7) Statement H4.1 "Appearance of the application interface is consistent between pages". The results of calculating the Likert score in statement H4.1 are presented in the following Table X:

Questions
The appearance of the application interface is consistent between pages. Based on the calculation of the Likert score in the calculation table H4.1, the value of the percentage of user satisfaction with the application display reaches 89.33% which means the user strongly agrees with the statement "Display the application interface is consistent between pages". 8) Statement H5.1 "The application can display a message when there is an error or error in using the application". The results of calculating the Likert score in statement H5.1 are presented in the following Table XI:

Questions
The application can display messages when there are errors or errors in using the application. Based on the calculation of the Likert score in the calculation table H5.1, the value of the percentage of user satisfaction with the application display reaches 89.33% which means the user strongly agrees with the statement "The application can display messages when there are errors or errors in the use of the application". 9) Statement H6.1 "There are recommended features to confirm the work you want to do on the application (example: delete data, edit data, etc.)". The results of calculating the Likert score in statement H6.1 are presented in the following Table XII:

Questions
The recommended feature is to confirm the work you want to do on the application (example: delete data, edit data, etc.). Based on the calculation of the Likert score in the calculation table H6.1, the value of the percentage of user satisfaction with the appearance of the application reaches 86% which means the user strongly agrees with the statement "There are recommended features to confirm the work to be done on the application (example: delete data, edit data, and others)".

Value
10) Statement H6.2 "There is a feature to return to the main menu without having to repeat the process that has been passed (for example: when the contents fill out but not finished, there is no need to press the back button repeatedly / undo function)". The calculated Likert score on statement H6.2 is presented in the following Table XIII:   TABLE XIII  CALCULATION RESULTS H6.2

Questions
The feature returns to the main menu without having to repeat the process that has been passed (for example: when the contents are not finished, there is no need to press the back button repeatedly / undo function). Based on the calculation of the Likert score in the calculation table H6.2, the value of the percentage of user satisfaction with the application display reaches 86.67% which means the user strongly agrees with the statement "The feature returns to the main menu without having to repeat the process that has been passed (example: when filling out the questionnaire but not finished, no need to press the back button repeatedly / undo function) ". 11) Statement H7.1 "Quick response to opening applications". The results of calculating the Likert score in statement H7.1 are presented in the following Table XIV:

Questions
The

Questions
Aesthetic display of up-to-date designs. Based on the calculation of the Likert score in the calculation table H8.6, the value of the percentage of user satisfaction with the application display reaches 86.67% which means that the user strongly agrees with the statement "The display is easy to read, understand and understand".
19) Statement H9.1 "The application provides a solution when an error occurs". The results of calculating the Likert score in statement H9.1 are presented in the following Table XXII:

Questions
The application provides a solution when an error occurs.  Based on the calculation of the Likert score in the calculation table H10.1, the value of the percentage of user satisfaction with the application display reaches 87.33% which means the user strongly agrees with the statement "Application provides help features".

V. CONCLUSION
The conclusion that can be drawn from this study is the E-Questionnaire website application can produce PBM questionnaire instruments (lecturers, students, alumni, and users of graduates/stakeholders) that are easy to use by respondents. E-Questionnaire website application can produce data that is used for PBM data processing that is responsive using Bootstrap Framework and CSS with Research and Development methods. All percentage interval indexes in the Heuristic Evaluation analysis model reach the category of strongly agree (H1.

THANK-YOU NOTE
Thank you to the Batam State Polytechnic community for their opportunity to become a case study of this research.