E-mail: ibrahim.eskandar@outlook.my
E-mail: syzul@yahoo.com
Mohd Syukri Yeoh Abdullah
Institute of Malay World & Civilization (ATMA), The National
University of Malaysia, Malaysia
E-mail: syukri@ukm.edu.my
School of Human Development and Techno-communication, University
Malaysia Perlis School of Human Development and Techno-communication,
University Malaysia Perlis, Malaysia
E-mail: amaniali@unimap.edu.my
E-mail: omar.mazni@gmail.com
Submission: 09/10/2018
Revision: 11/06/2018
Accept: 11/06/2018
ABSTRACT
There
are major concerns regarding the previous research that can be said as having
poor measurements, lack of theoretical grounding, and concern heavily on
investment justifies. Some are also considered as having weak survey
instruments, inappropriate data collection approaches, and lack of agreement on
the dependent variables measurement that resulted in its incompatibility.
Several related issues pertaining to high rates of systems’ failure that
justify the heavy investment costs and affect the systems’ success of
measurement. This matter had raised a high concern, especially for the
researchers, practitioners, organization managers and systems administrators.
Thus, the successful measurement of any systems is vital. The purpose of this
study is to provide a framework and a high-quality validated instrument based
on the literature review and experts’ validation. Subsequently, it can be used
in the future studies to efficiently assist in the systems’ success
measurement. The statistical test of 344 users with the smart PLS for the
instrument shows an excellent result.
Keywords: Experts’ Validation; Systems Success;
Systems Fail
1. INTRODUCTION
In the Arab
world, especially Yemen having almost nonexistence studies regarding to
information systems success (FADHEL et al., 2018a; FADHEL et al., 2018b). The
users of the systems are expanding than ever before, this result in measurement
complexity of systems success. Researchers nowadays are facing many challenges,
which is the sophistication of systems. With the increasing number users, this
phenomenon can make us lose sight of the key elements such as (relevance,
accuracy, timeliness etc.) of quality that are playing a major role in the
success of the systems. The growth of measurements of systems success is
increasing, which leads to more complexity that needs future studies (DELONE;
MCLEAN, 2016).
For
the researchers, DeLone and McLean (2016) highlight several themes that can be
studied in future. One of these highlights is developing an adaptive research
processes and frameworks to measure systems success. There is still a lack of
development approach for building web systems success quality framework, and
the field of measuring the web-based systems is not yet mature (ZAHRAN et al.,
2014). Similarly, success measure is necessary to help assess how the systems
is performing and outline any issues that could possibly be causing hindrance
for the users (DELONE; MCLEAN, 2016).
2. BACKGROUND
Various
conceptual frameworks and empirically instruments that have been tested,
suggested to-date, each of them highlighting different factors that could
potentially underline how a good quality WIS can be built. However, majority of
these frameworks such as DeLone and McLean (1992), WEBQUAL and W-QEM are more
suitable for business purposes (KHRED, 2017; MEBRATE, 2010). Sadly, most of
these frameworks and instruments are in limited domain (MWANGI, 2016) and have
not been thoroughly tested or applied in connection to system website
development and implementation. Most of the frameworks and instruments only
provide general quality characteristics, which is not helpful at times and
seems to be a waste of resource as well (MEBRATE, 2010).
No
mutually agreed definition is available and/or any reliable measurement
instruments (MARDIANA; TJAKRAATMADJA; APRIANINGSIH, 2015; MCNAB; LADD, 2014;
MWANGI, 2016). Therefore, the basic concerns are still prevailing pertaining to
the explanation of quality criterion that could be potentially deployed to
examine systems quality and effectiveness (MWANGI, 2016).
Accordingly,
mixed results have also been reported in terms of what can explain systems.
Mostly they outlined in relation to systems and software’s includes ineffective
measurements, limited theoretical grounding, reliance on financial performance,
lack of data collection, and limited knowledge on prediction (MARDIANA et al.,
2015; DELONE; MCLEAN, 2016). While many studies have investigated the
relationship between information systems (IS) characteristics and IS use, the
results have been inconsistent (FORSGREN et al., 2016). This severe mixed
results and lack of empirical focus on systems outlines psychological,
cognitive and passionate prospects, which could intervene the relationship
between the instruments and predictors of success (SNEAD et al., 2015). Research in the relationship of information systems
success (use, satisfaction and benefit) has produced mixed results (GOEKE;
CROWNE; LAKER, 2018).
Systems in the organizations are
still having confrontation of the lack of research targeted users’ satisfaction
(LAUMER, 2016; POLITES; KARAHANNA, 2012). Business researchers have been
targeted the satisfaction of the users in the context of business.
Nevertheless, notwithstanding the fact that the higher education market is
getting more discerning, there is noteworthy lack of research (WONG, 2016).
3. INSTRUMENT OF
SUCCESS MEASUREMENT
The current
instrument consists of 11 constructs: 8 independent variables (IVs) (information
quality, systems quality, ease of use, security, usability, reliability,
functionally and efficiency); and 3 dependent variables (DVs) (satisfaction,
benefit and loyalty).
3.1.
Information
quality
In the views of Edlund and
Lovquist (2012) as cited by referring to Petter et al. (2008) quality in
connection to information refers to significance and value of the provided
information generated by the IS exhibits. Therein, with regards to measuring,
how much satisfied the end users are with the provide information and its
quality becomes the major important factor. As a consequence, it is viewed as
the most important factor to outline user satisfaction which is often not found
in an appropriate manner.
Accordingly,
Edlund and Lovquist (2012) have also asserted that IS quality in terms of the
information provided also defines the end-user satisfaction and thus, guides as
to what length, it is reaching up to its expectations. Notably, user may end up
experiencing frustration, if they fail to achieve accurate and quality
information from the provided IS. Edlund and Lövquist (2012) based on
the assertions of Bharati and Berg (2005) and Petter et al. (2008) have also
outlined that several incidents can be noted pertaining to arguments amongst
prominent researchers in this domain regarding the significance of information
and its quality in particular. Studies have outlined several factors that
define quality of information, which includes but not limited to accuracy,
precision, relevance of information, element of timeliness and completeness.
Information quality:
refers to the prominent features of the piece of outcome from a system or
entity. In detail, this denotes to completeness, understandability and
accuracy. On the grounds of the critical appraisal of the literature,
information quality is highly important for student`s satisfaction when it
comes to the website of the university.
First
IV Perceived Information Quality (PINFQ) measures the accuracy, content and
understandability:
a) The
information outputs of my university web system (including on-screen and
printed outputs) are Complete.
b) The
information outputs of my university web system (including on-screen and
printed outputs) are concise and are easy to understand.
c) It is
easy to find what I’m looking for when using my university web system.
d) The
information outputs of my university web system (including on-screen and
printed outputs) are accurate and is free from errors.
e) My
university web system provides the precise information I need.
These
questions adapted from (BYRD et al., 2006; CHEN; KAO, 2012; CHIU et al., 2016;
DAVARPANAH; MOHAMED, 2013; EDLUND; LÖVQUIST, 2012; FADHEL, 2015; GORLA; SOMERS;
WONG, 2010; MOHAMMADI, 2015; WANG; LIAO, 2008; ZAIED, 2012).
3.2.
SYSTEM
QUALITY
Edlund
and Lövquist (2012) whilst quoting
Bharati and Chaudhury (2004) has outlined system quality as the generic
performance of the information system. Whilst referring to Delone and McLean
(1992) and Edlund and Lovquist (2012) highlighted another explanation of system
quality. According to them, system quality talks about attributes of an
information system that ensures the generation of information that is valuable
for making effective decisions.
In
connection to the study by Petter et al., (2008) vital prospects pertaining to
system quality talks about system flexibility; ease of learning, and ease of
use. This refers to extent to which the usage and learning of a particular
system requires no effort and hassle. This element is crucial in the prospect
of information quality since such efforts vary in terms of how they are
perceived by the users. Individuals viewing a particular system as requiring
more effort and full of stress may result in avoiding the use of the system.
Therefore, IS usability perceptions being essential in this regard (EDLUND;
LÖVQUIST, 2012).
System
quality denotes to noteworthy features of the system including adaptability, trust
and sophistication. On the basis of the literature review system quality, it is
also principally related with student’s satisfaction with the web systems.
The Second IV Perceived System
Quality (PSYSQ) measures the adaptability and sophistication:
a)
It is easy for me to become skillful by using my
university’s web system.
b)
In general, I find my university’s web system is easy
to use.
c)
My university’s web system is well integrated.
d)
My university’s web system has a short time lag
between input and output of data, as example ‘the registration process’.
e) My
university’s web system has a short response time for on-line enquiry.
Those
questions were adopted from (CHIU et al., 2016; FADHEL, 2015; GORLA; SOMERS;
WONG, 2010; MOHAMMADI, 2015; ZAIED, 2012).
3.3.
EASE
OF USE
According
to Ofori, Larbi-Siaw, Fianu, Gladjah and Boateng (2016), as per the
explanations by Davis (1989), it mentioned that Perceived Ease of Use (PEOU)
denotes to the extent to which an individual believes that the specific
information system or technology would be effort-free in use. Davis (1989) has
also asserted that technology and its usefulness relies upon how convenient it
is for users to use. In a simpler term, the easier it is for the users, the
important it would be for them to interact with the social media, web portals
and other online platforms cited in (OFORI et al., 2016).
Jongchul
and Sung-Joon (2014) and Park, Rhoads, Hou and Lee (2014) on the other hand
stated that there is a causal connection between PEOU and PU. This connection
has also been confirmed by several studies conducted in different occupation
settings cited in (OFORI et al., 2016). Al-Azawei and Lundqvist (2015) whilst
referring to Venkatesh and Davis (2000), outlined that perceived ease of use
refers to the extent to which a user views the usage of a particular system
would be convenient and free from all the hassles and efforts. Thus, technology
acceptance model 2 (TAM2) as elaborated in PEOU is important in outlining
perceived usefulness and users’ attitudes towards a technology.
Perceived
ease of use: The term denotes to student perception about the website
usefulness and ease in connection to physical efforts. Prominent literature has
sketched a significant association of ease of use of students as users with
university`s electronic web systems.
Third IV Perceived Ease of Use
(PEOU) measures the systems easiness:
a)
I find my university’s web system flexibility to
interact with.
b)
My interactions with my university’s web system during
an online process were clear and understandable.
c)
My university’s web system is convenient for me.
d)
My university’s web system is laid out in a modern and
fashionable.
These questions were adopted from
(DEVARAJ et al., 2002; KHAWAJA; BOKHARI, 2010; LIU et al., 2010; MOHAMMADI, 2015;
WOLFINBARGER; GILLY, 2003).
3.4.
RELIABILITY
According
to Dreheeb et al., (2016) and Selvakumar, (2016) reliability is also an
important and essential prospect when it comes to software quality. Dohi and
Nakagawa (2013) however, mentioned that the reliability is set of attributes
that can potentially trigger individual capability to maintain its performance
level in a given period of time. Therein, the system is required to keep hold
of software faults to ensure reliability and minimize software crashes. The
systems are typically capable of re-establishing their performance levels in
order to carry on generating same results based on Papanikolaou and
Mavromoustakos (2008).
Reliability in connection to IT
refers to capability of a system to offer or provide designated functions and
features in a particular time period (MBIWA, 2014). Accordingly, Shiratuddin
(2015) had suggested that the degree to which, a product and/or component
executes the outlined conditions as per the specifications. Some
of the reliability prospects are concerned with acting upon elements necessary
for promised timings (VAN IWAARDEN et al., 2004).
Reliability
denotes to the extent to which the system features and prospects are robust to
perform specific functions and provide designated services and outcomes.
Reliability includes maturity, fault tolerance, recoverability and
availability. Literature has supported a significant relationship between
reliability on the use of student`s satisfaction when it comes to the web
system usage.
Forth
IV Perceived Reliability (PREL) measures the maturity, fault tolerance,
recoverability, availability and reliability:
a) My
university’s web system never stops unexpectedly.
b) When
there is a problem in some part or parts in my university’s web system, I can
still can browse and perform some of processes.
c) In
case of interruption of faulty, my university’s web system will recovers in a
timely manner.
d) In
general, my university’s web system is available 24 hours.
e) I
believe that my university’s web system is reliable.
Those
questions were adopted from (AGHAZADEH et al., 2015; ALVES ET AL., 2015;
CONSTANTIN, 2013; DEVARAJ; FAN; KOHLI, 2002; MEBRATE, 2010).
3.5.
USABILITY
Al-Manasra,
Khair, Zaid and Taher Qutaishat (2013) said usability
is an important. It is also one of the most important factors to outline
software quality (DREHEEB; BASIR; FABIL, 2016). Usability is a crucial
component that relies upon how well a particular application and/or software.
According
to Madan and Dubey (2012) usability outlines crucial attributes pertaining to
the establishment of successful software applications cited in (DREHEEB; BASIR;
FABIL, 2016). Likewise, Dreheeb, Basir, and Fabil (2016) have also asserted
that e-learning success is essential, and it is only possible through
responsive usability of the software and online features (ARDITO et al., 2006).
Moreover, usability is the core premise for the evaluation of e-learning
technologies and systems. It denotes to the considerable features and prospects
of software that enables it to help users understand, learn and attract
connotations under specified conditions (DREHEEB; BASIR; FABIL, 2016; CHO;
HYUN, 2016).
Usability
defines as the extent to which, product or systems can bring it is feasible and
objective to provide specific objectives, and thus it facilitates in achieving
effectiveness and efficiency in satisfaction in the context of the use of any
specified system. Usability includes user interface aesthetics and protection
from user's error. Past studies have also outlined a strong association of
usability perceptions with student`s satisfaction with web-site systems.
Fifth
IV Perceived Usability (PUSA) measures the user interface aesthetics and
protection from users’ error:
a) The
interface design of my university’s web system is attractive.
b) All
interface elements are well combined and harmonious in my university’s web
system.
c) My
university’s web system protects me from making errors when interring data.
d) My
university web system errors messages clearly indicate to me how to rectify the
problem.
e) In my
university’s web system, it is easy to recover from the error instantaneously.
Those
questions were adopted from (ALVES et al., 2015; ASTANI; ELHINDI, 2008;
MEBRATE, 2010; PADAYACHEE; KOTZE; VAN DER MERWE, 2010; WOLFINBARGER &;
GILLY, 2003; SUWAWI, 2015).
3.6.
FUNCTIONALITY
Functional
prospects refer to the potential of a service or product meeting the implied
needs under particular conditions (MBIWA, 2014;
TANDON; KIRAN; SAH, 2017). Also referred
as Suitability, functionality is the degree to which a particular product or
systems offers processes and functionality that meets the desired expectations
of the customers (SHIRATUDDIN, 2015).
Functionality
defines as the extent to which, a specific product or service-based system
offers features and prospects that are in line with the implied needs of users
under designated conditions. Functionality includes navigation and search.
Literature on the topic has also underlined functionality to be of high
significance when it comes to student`s satisfaction, whilst using university’s
web systems and related systems.
Sixth
IV Perceived Functionality (PFUN) measures the navigation and search:
a) It is
easy to go to the home page while I’m browsing any other page in my
university’s web system.
b) While
using my university’s web system, I can easily navigate backwards through
previously visited pages.
c) My
university’s web system provides varied search options (e.g. by faculty,
courses, etc.).
d) Search
hints are provided when wrong keywords search is used.
Those
questions were adopted from (ALADWANI,
2002; KHAWAJA; BOKHARI, 2010; MEBRATE, 2010).
3.7.
EFFICIENCY
Aghazadeh,
Pirnejad, Aliev and Moradkhani (2015) stated that efficiency is another
important component which refers to performance quality of the software. As per
definition, efficiency in connection to systems can be referred as capability
of the software towards offering responsive performance, whilst using highly
reasonable amount of resources in any stated situation.
Users
are generally expected to operate in a manner whereby, they have to use the
minimum amount of resources with the highest possible e-learning experience.
Accordingly, system response denotes to performance of the system or software
in terms of time, graphics, page set up and loading in order to enhance the
user satisfaction (PAPANIKOLAOU; MAVROMOUSTAKOS, 2008) as in (DREHEEB et al.,
2016). Efficiency is also very important when it comes to performance of the
software and to what length it is relatively using minimum resources compared
to other alternative options in a given situation (MBIWA, 2014).
Efficiency
defines as the potential of the software to offer desired functions in order to
reach the desired objective needs from the software use. Efficiency includes
time behavior and accessibility. Literature available on the topic has also
confirmed its significance relationship with user`s satisfaction with the web system
of the university.
Seventh
IV Perceived Efficiency (PEFF) measures the time behavior and accessibility:
a) It is
possible to find in my university’s web system what I want in a reasonable
time.
b) My
university’s web system enables me to get on to it quickly.
c) My
university’s web system does not use advertises or unwanted plug-ins.
d) I can
access my university’s web system from my favorite browser.
e) It is
easy to get and browse any part on my university’s web system.
Those
questions were adopted from (ALVES et al., 2015; KHAWAJA; BOKHARI, 2010;
MEBRATE, 2010; ROCHA, 2012; ZEHIR et al., 2014).
3.8.
SECURITY
When
it comes to the web systems, their technical features may include a security.
As per definition, Astani and Elhindi (2008) had suggested that the web system
security relates with authentication for user and its potential in this regard.
In detail, security refers to the capability of a portal to provide secure
access virtual environment to users whereby, they can use data related to a
given product or service without any scam (MBIWA, 2014).
According
to Shiratuddin (2015) and Ludin
and Cheng (2014) security also
denotes to degree to which the system protects the information and data in such
a manner that users are able to access it as per the level of authorization.
Security effects the satisfaction of users significantly (CHIANG;
HUANG; YANG, 2011).
Security
defines as to caters to privacy in the mutual exchanges i-e financial as well
as non-financial. The availability of secure inline systems builds students`
confidence and reliability in the web portal and offers a friendly environment
for completing transactions, which includes security and privacy. The extent to
which system protects information and important data related to personnel
involved in the transaction significantly enhances user satisfaction with the
web system of the university.
Eighth
IV Perceived Security (PSEC) measures the security privacy and trust:
a) I
believe my university’s web system is secure.
b) Overall,
I trust my university’s web system.
c) My
university’s web system has adequate security features that make you feel
secure while using.
d) I
believe that the information offered by my university on the university’s web
system is sincere and honest.
e) The
output information of my university’s web system is secure.
These
questions were adopted from (ALVES et al., 2015; JEON, 2009; MALIK et al.,
2016; WEBB; WEBB, 2004; WOLFINBARGER; GILLY, 2003; ZAID, 2012; ZEHIR, 2014).
3.9.
SATISFACTION
In
the views of Vaezi et al., (2016) the literature on the topic has outlined a
considerable gap in terms of what user desire and expect against what is
offered to them and how it influences success of an information system.
Overall, there are several explanations and instrument for measurement are
currently available, but how and to what length these measures are vital may
varies. Gradually, the scholars are moving towards developing one unified user
satisfaction construct and measure, which would ultimately change the idea and
view about the concept especially in the area of information systems research
(VAEZI et al., 2016).
Studying
user satisfaction and how to predict it is essential for organization. Such
studies are significant to help enterprises comprehend with the idea and how
they can strategize to keep their users satisfied with their services,
facilities and performance prospects (VAEZI et al., 2016). Concerning the
dimensions of success, satisfaction of users is essential due to the fact that
the field of research pertaining to Information System (IS) is very limited.
Students’
satisfaction denotes to the measure of satisfaction of students with the major
system features a student interacts with. This primarily includes online
support systems, reports and access, university online systems and online
course data banks. Review of the literature has suggested that satisfaction to
student with system and online portals can be of significant value towards
system benefit and enhancing loyalty with these web systems.
First
DV Students’ Satisfaction (STSA) measures the students’ satisfaction
a) My
university’s web system is of high quality.
b) My
university’s web system has met my expectations.
c) My
interaction with my university’s web system is very satisfying.
d) Overall,
I am satisfied by using my university’s web system.
e) Overall,
I’m happy with my university’s web system.
Those
questions were adopted from (AL-AZAWEI; LUNDQVIST, 2015; CHIU et al., 2016;
CONSTANTIN, 2013; EPPLER; ALGESHEIMER; DIMPFEL, 2003; FADHEL, 2015; JEON, 2009;
KIRAN; DILJIT, 2011; LIAW; HUANG, 2013; MOHAMMADI, 2015).
3.10.
BENEFIT
Wang
and Liao (2008) showed the result that has considerable support for the DM
model and encourage the study of perceived net benefit. Alshibly (2015) also
forwarded support and recommendations towards net benefits and asserted that it
ideally should be designed under a specific framework to help scholars and
practitioners to effectively assess system benefits. DeLone and McLean (2016)
have outlined that some of the most prominent measures for assessing IS success
are designers, managers, users and so on. Therein, the net impacts are system
outcomes, which are generally compared to the core purpose of the system.
For
this reason, the Net Impacts construct will be the most contextual dependent
and varied of the six D&M Model success dimensions (DELONE; MCLEAN, 2016).
Several methods are available to examine the net effects at all four levels i-e
organizational, individual, and societal and industry. It is recommended that
the usage of individual measure would be more appropriate for the assessing
information system success, rather than from other general prospects (DELONE;
MCLEAN, 2016).
Benefit
defines as one of the highly important prospects of systems success is the
benefit measure, which denotes to the influence and outcomes of the systems
from individuals to economies and societies at large. Scholars in the area have
outlined a significant feature when it comes to systems and their benefits. The
benefits refer to the extent to which a system is healthy and worthwhile for
users, organizations, groups, business sectors and economies at large such as
system facilitation in decision making, productivity enhancement, welfare or
job effectiveness.
Second
DV Benefit (BENE) measures the systems benefit
a) My
university’s web system helps me to retrieve my information easier and quickly.
b) My
university’s web system saves my time.
c) Overall,
I obtained benefits from using my university’s web system.
d) My
university’s web system is an important and valuable aid to me.
e) My
university’s web system has a large, positive impact on me as a user.
Those
questions were adopted from (CHIU et al., 2016; DERNBECHER, 2014; FADHEL, 2015;
MCGILL; HOBBS; KLOBAS, 2003; WANG; LIAO, 2008; WIXOM; WATSON, 2001).
3.11.
LOYALTY
There
seems to be little empirical attention towards outlining what causes customer
satisfaction, especially amongst the tertiary students and whether or not,
these service features are capable of generating healthy benefits and outcomes
such as customer satisfaction and loyalty towards institutions (BROWN;
MAZZAROL, 2006). Brown and Mazzarol (2006) have also asserted that at present
educational environment is more regarded as service business and students as
customers. A study made by Senate (2001) outlined that the customer
satisfaction and customer value, both have values towards academic institutions
(BROWN; MAZZAROL, 2006).
Likewise,
Cronin, Brady, and Hult (2000) have mentioned that behavioral intentions as the
last items in the analysis. According to them, rising customer retention and
lower customer defection is the core prospect through which an organization can
generate more profits (ZEITHAML; BERRY; PARASURAMAN, 1996). In the views of
Cronin, Brady and Hult (2000), positive behavioral intentions are important for
enterprises and help them to get their customers to forward positive thoughts
about the service and company products like positive words; recommendations;
express loyalty; investing more in other company products and show willingness
to purchase premium products.
Loyalty
defines as a behavioral prospect that outlines acceptance and satisfaction with
a certain product or service and leads towards repeat using, encourages
referrals and recommendations. Loyal students in this context would be ones
engaged in repeatedly using the online system of the university and actively
recommending of the same to other students.
Third
DV loyalty (LOYA) measures the students’ loyalty
a) I
will be using more of my university’s web system in the future.
b) I
will recommend my university’s web system to others.
c) I
will say positive things about my university’s web system to others.
d) I
like using my university’s web system.
e) I use
my university’s web system frequently.
Those
questions were adopted from (CONSTANTIN, 2013; EPPLER et al., 2003; JEON, 2009;
KIRAN; DILJIT, 2011; MOHAMMADI, 2015; VALVI; WEST, 2013; ZEHIR et al., 2014).
4. PROBLEM AND OBJECTIVE
With
the high number of the systems failure existed globally, they are due to the
mix results and weak survey instruments (FADHEL et al., 2018a; FADHEL et
al., 2018b). Thus, this study aims to produce a new prescription
for the systems success measurement by providing a high reliability validated
instrument and framework.
5. FLOW AND METHODOLOGY
Yemeni
universities web systems are large integrated applications that considered as
primary central applications for student information. It allows the
administrator to manage and provide data to the staffs, students visitors, etc.
Furthermore, it is giving permission to the students to register and deal
within the related details of the study until they graduated (KHRED, 2017).
This
study starts with the validation process for the instrument, which is a strong
way for effectively instrument design. Based on the previous studies, to make
the process of validation in the fields of information systems and software
engineering (quality and testing), there must be at least three experts
(academic experts in the field or in the related fields with a PhD as a minimum
qualification or technical experts in the field with at least 3 years of
experience).
Then,
this study used all the required statistical tests for instrument approval as a
valid tool to measure web-based system's success in the domain of
universities. A pilot test was performed
all necessary statistical tests to measure the instrument reliability has been
done (Rho_A, Composite Reliability (CR), Average Variance Extracted (AVE) and
Cronbach’s Alpha (α)). Confirmatory Factor Analysis (CFA) has also been done to
see how were items load, are those items related to their constructs are not.
Finally,
after real data collected the required statistical tests has been performed
Construct Validity, Convergent Validity, Discriminant Validity and
Multicollinearity (CFA, Rho_A, CR, AVE, α, Fornell, HTMT and VIF). After making
sure that all tests are perfect the instrument and framework are proposed.
Research
can be either be qualitative, quantitative or mixed methods. The best method
depends on the research purpose as each research has its own merits and
demerits (FADHEL, 2015). This study aims to comprehensively explain the
phenomenon by using a quantitative &
qualitative methods to achieve the
maximum benefits, these approaches are considered as the best means that are
suited under the current circumstances. Current research following the
qualitative way in the process of the instrument validation only.
After
the validation process ended a full quantitative way has been used in testing
the instrument in the pilot test and in the real data test. Instrument has been
tested in pre-test with 9 users, pilot test with 33 users then used in final
data collection process with 344 students (users of the systems) from three
different universities. Smart PLS used to perform the
results as its categorized as one of the best tools used for predicating the
results.
Based on the literature review of
Smart PLS, we provide a summary of its benefits, that are it
works well with structural equation that are comparatively new techniques to
model series of cause-and-effect connections with latent variables. The PLS-SEM
method is known to be a user-friendly tool for statistical model development in
addition to predicting or making forecast cited in. Particularly, it was
employed for the study because of the following reasons. To develop an
instrument and a structural equation models was the first motives PLS-SEM was
leveraged. However, it has been illustrated to be advanced in performing models
in the fields of information systems and software engineering.
Additionally, it is more suitable
for the actual world applications, as it is well more beneficial when modeling.
PLS soft modelling technique includes (i.e. ability to flexibly develop and
validate complex models) it can also be employed to estimate huge complex
models. PLS-SEM application on huge complex models is the main reason the study
adopts PLS for enhanced prediction. In many existing information systems and
software engineering researches, data tends to possess normality issues.
However, PLS does not necessarily needs the data to be normal for it to be
analysed. Also, non-normal dataset is treated better with PLS.
To avoid problem of data normality
path modelling technique was finely selected. Moreover, PLS used in behavioral
and social sciences, SEM is a powerful statistical analysis tools that is able
to test various relationships concurrently. Finally, PLS-SEM has a valid and
semantically correlated outcome, while other existing techniques for data
analysis often results in less unclear outcomes and mostly have separate
analyses.
6. QUALITATIVE RESULTS
In this research, experts reviewing
and consulting for this work, instrument and framework is one of the main
steps. Noted from the literature most of researchers used 4-7 experts in their
researches. Number of experts can be exceeding 20 experts, no matter if it is
exceeding twenty validators but usually minimum numbers is preferred. Typically, in most of researches number of
experts around six quoted in (OLSON, 2010) by his referring to (HOLBROOK et al., 2007;
JANSEN; HAK, 2005; PRESSER; BLAIR, 1994; THEIS et al., 2002).
6.1.
Expert
number one
6.2.
Expert
number two
6.3.
Expert
number three
The
next stage the researcher had contacted Dr. Ali, who is specialised in
information systems and works as Visiting Senior Lecturer in the Universiti
Utara Malaysia. In the first contact with Dr. Ali via telephone, the researcher
had asked for a pilot test, and told him that the pilot test will be performed
after completing the process of validations from all experts. He advised, to
perform the pilot test initially, since the the two prior experts had already
accepted the work.
He
then recommended to the researcher to perform
another pilot test once the validation process has been completed. The
researcher responded to Dr. Ali’s request and then met him in his office for
two hours and half. Dr. Ali’s expert advice had provided his agreement on the
work, with some adjustments in the instrument. The researcher finally applied
the adjustments and gets the agreement on the instrument.
6.4.
Expert
number four
On
first of November 2017, the researcher gets the validation agreement from
Professor Azizah, she is specialized in software engineering and works as a
lecturer in Universiti Utara Malaysia. Prof. Dr. Azizah in the second
face-to-face meeting provided her strong agreement upon the work, framework and
the questionnaire with an advice to remove one construct and some items.
6.5.
Expert
number five
After
finishing with the four experts, the researcher then call and send an email to
Professor Dr. Ahmed, he was the former Dean of Faculty of Engineering and
Computer Science and currently works as lecturer in Yemen. Strong and critique advices
have been provided by him to remove some constructs, with the agreement on the instrument
and framework.
6.6.
Expert
number six
Continually,
researcher referred to Miss Fawzia the Director of Systems and Information for
more than ten years in Yemen. She agreed
with the advices given by Professor Dr. Ahmed, and she then provided the
same advices for removing constructs and some items to make the framework
comprehensively related to systems quality. Finally, she gives the researcher
her covenants with advice to contact Dr. Fathya on the quantitative specialized
area.
6.7.
Expert
number seven
The
researcher had applied to Miss Fawzia advices and directly communicated with
Associated Professor Dr. Fathya, in Yemen. Dr. Fathya is specialized in
quantitative science. She provided a novel advice regarding to the Likert scale
and removing the constructs that are not related to quality of systems. She
also, provided advices regrading to data analysis and how unnecessary
constructs and items can be negatively affect. She said she sees the signs of
success and she sent her agreement via email.
6.8.
Expert
number eight
After
all these advices researcher applied all the notes and remove the unrelated
constructs and items from the framework. The instrument went to the respondents
for a new pilot test and the results was brilliant. Before went to the main
data process collection. The researcher had communicated with Dr. Israr from
India.
The
final validator for this research is Dr. Israr, he was working as Assistant
Professor in Yemeni universities and with university Jamia Millia Islamia in
India. His specialization is in systems and computer science. Dr. Israr sent
his agreement with advice of using PLS as the analysing tool. Finally, after
applied all experts’ notes, pre-test and pilot test, main data collection
process was performed confidently.
7. QUANTITATIVE RESULTS
Here
the researcher provides the results based on the pilot test and real data
collection.
7.1.
Pilot
test
Significance
of a pilot test in research contemplate can never be overemphasized claiming it
diminishes the pressure that the researcher could have experienced amid the
final analysis of the research (CAVANA
et al.,
2001).
Subsequently, it is extremely urgent to lead a pilot test to assist researcher
in assembling a decent establishment for the significant examination (ADEBOLA,
2014).
The
pith of the pilot contemplate is to assist the researcher to find pressing
issues that may emerge from the questionnaires and by allowing researchers in
readdressing and altering principal consideration in the questionnaire (ADEBOLA,
2014; PALLANT, 2007).
Pilot
test results are illustrated in the form of the tables. The tables show that
all tests are acceptable and the values are accepted and showed excellent
reliability. The factors loading for all items are perfect and all items under
its related construct.
Table 1: Pilot
Result - Construct Reliability and Validity
Rho_A |
Composite
Reliability |
Average
Variance Extracted |
Cronbach’s
Alpha |
|
Benefit |
0.8191 |
0.8726 |
0.5784 |
0.8177 |
Ease of Use |
0.7914 |
0.855 |
0.5976 |
0.7734 |
Efficiency |
0.8076 |
0.8635 |
0.5597 |
0.8019 |
Functionality |
0.8239 |
0.8769 |
0.6435 |
0.8097 |
Information Quality |
0.854 |
0.895 |
0.6317 |
0.8520 |
Loyalty |
0.8018 |
0.8633 |
0.5591 |
0.8011 |
Reliability |
0.7938 |
0.8569 |
0.546 |
0.7905 |
Satisfaction |
0.829 |
0.8749 |
0.5857 |
0.8189 |
Security |
0.8113 |
0.8682 |
0.5693 |
0.8097 |
System Quality |
0.838 |
0.8827 |
0.6023 |
0.8328 |
Usability |
0.8256 |
0.8741 |
0.5825 |
0.8191 |
Source: The Researcher
Table 2: Pilot
Result - Confirmatory Factor Analysis
Factors |
Items |
Loadings |
Benefit |
Benefit1 |
0.7713 |
Benefit2 |
0.7653 |
|
Benefit3 |
0.7964 |
|
Benefit4 |
0.7162 |
|
Benefit5 |
0.7511 |
|
Perceived Efficiency |
EFF1 |
0.7444 |
EFF2 |
0.7812 |
|
EFF3 |
0.7823 |
|
EFF4 |
0.7771 |
|
EFF5 |
0.6468 |
|
Perceived Ease of Use |
EU1 |
0.8162 |
EU2 |
0.7113 |
|
EU3 |
0.8532 |
|
Eu4 |
0.7002 |
|
Perceived Functionality |
FUN1 |
0.8772 |
FUN2 |
0.8158 |
|
FUN3 |
0.8534 |
|
FUN4 |
0.6407 |
|
Perceived Information Quality |
IQ1 |
0.676 |
IQ2 |
0.8361 |
|
IQ3 |
0.8372 |
|
IQ4 |
0.8206 |
|
IQ5 |
0.7927 |
|
Loyalty |
Loy1 |
0.6816 |
Loy2 |
0.6967 |
|
Loy3 |
0.7908 |
|
Loy4 |
0.7971 |
|
Loy5 |
0.7648 |
|
Perceived Reliability |
REL1 |
0.7315 |
REL2 |
0.6623 |
|
REL3 |
0.7302 |
|
REL4 |
0.8083 |
|
REL5 |
0.7547 |
|
Perceived System Quality |
SQ1 |
0.7099 |
SQ2 |
0.815 |
|
SQ3 |
0.7908 |
|
SQ4 |
0.6924 |
|
SQ5 |
0.8595 |
|
Satisfaction |
Satisf1 |
0.6681 |
Satisf2 |
0.7209 |
|
Satisf3 |
0.9047 |
|
Satisf4 |
0.7517 |
|
Satisf5 |
0.7609 |
|
Perceived Security |
Sec1 |
0.7425 |
Sec2 |
0.7372 |
|
Sec3 |
0.784 |
|
Sec4 |
0.6852 |
|
Sec5 |
0.8172 |
|
Perceived Usability |
USab1 |
0.6997 |
USab2 |
0.8235 |
|
USab3 |
0.8221 |
|
USab4 |
0.7216 |
|
USab5 |
0.7404 |
Source: The Researcher
7.2.
Real
data result
This
phase of research shows the consequences of investigatory analysis of the
research study using PLS principal component analysis. Every scaled construct
for the propose study was concurrently adapted from existing researches. As
reported in (UNIT,
2013) If the
construct loadings greater or equal to 0.6 this construct is reliable without
attention to size of sample (GUADAGNOLI;
VELICER, 1988).
The
statement of Guadagnoli and Velicer (1988) is advocated and supported by (FIELD,
2005). Cut-off
should be used with items of 0.4 loading without care to size of sample (STEVENS,
1992). Loading
of 0.32 is poor,0.45 is fair, 0.55 is good, 0.63 is very good and 0.71 is
categorized as excellent loading (TABACHNICK;
FIDELL, 2007).
Construct items should be 0.6 and above to perform reliable analysis especially
if sample size is small (MACCALLUM et al., 2001).
Here in the tables real data test results of 344 users are illustrated. Tables
showed all tests are acceptable and the values were perfect.
Table 3: Real Data Result - Confirmatory Factor
Analysis
Factors |
Items |
Loadings |
Benefit |
Benefit1 |
0.7540 |
Benefit2 |
0.6136 |
|
Benefit3 |
0.7504 |
|
Benefit4 |
0.7363 |
|
Benefit5 |
0.7253 |
|
Perceived Efficiency |
EFF1 |
0.7032 |
EFF2 |
0.7287 |
|
EFF3 |
0.6874 |
|
EFF4 |
0.7525 |
|
EFF5 |
0.6104 |
|
Perceived Ease of Use |
EU1 |
0.8021 |
EU2 |
0.6987 |
|
EU3 |
0.7923 |
|
Eu4 |
0.7079 |
|
Perceived Functionality |
FUN1 |
0.7575 |
FUN2 |
0.7347 |
|
FUN3 |
0.7610 |
|
FUN4 |
0.7742 |
|
Perceived Information Quality |
IQ1 |
0.7138 |
IQ2 |
0.7419 |
|
IQ3 |
0.7276 |
|
IQ4 |
0.7048 |
|
IQ5 |
0.7037 |
|
Loyalty |
Loy1 |
0.7045 |
Loy2 |
0.7614 |
|
Loy3 |
0.7911 |
|
Loy4 |
0.7891 |
|
Loy5 |
0.7700 |
|
Perceived Reliability |
REL1 |
0.6747 |
REL2 |
0.7059 |
|
REL3 |
0.7055 |
|
REL4 |
0.7213 |
|
REL5 |
0.7035 |
|
Perceived System Quality |
SQ1 |
0.6723 |
SQ2 |
0.7661 |
|
SQ3 |
0.686 |
|
SQ4 |
0.7542 |
|
SQ5 |
0.7354 |
|
Satisfaction |
Satisf1 |
0.7094 |
Satisf2 |
0.7601 |
|
Satisf3 |
0.7438 |
|
Satisf4 |
0.7350 |
|
Satisf5 |
0.7256 |
|
Perceived Security |
Sec1 |
0.7468 |
Sec2 |
0.7982 |
|
Sec3 |
0.7174 |
|
Sec4 |
0.6505 |
|
Sec5 |
0.6763 |
|
Perceived Usability |
USab1 |
0.7582 |
USab2 |
0.7960 |
|
USab3 |
0.7839 |
|
USab4 |
0.8075 |
|
USab5 |
0.7002 |
Source: The Researcher
7.3.
Construct
Validity
Construct
validity construct validity evaluates the degree gotten from employing a
measure using fit of theories where test is planned (SEKARAN; BOUGIE, 2010).
More so, it is worried about responding to inquiry: does the research
instrument identify concept as theorized? In accomplishing the validity test,
two kinds of validity tests were subjected to the scales of measurement:
(convergent validity) and discriminant validity (DYBA, 2005). Two sub-classes of construct validity are convergent
and discriminant validity (SEKARAN, 2003). Also, Hair et al., (2017) proposed average variance extracted (AVE) to evaluate
convergent validity.
Convergent
validity of this study was measured by methods for normal difference separated
method or (average variance extracted technique AVE). AVE is the normal
difference shared amongst variable and its measures. AVE variable ought to be
greater than the fluctuation shared amongst variable with other variables in a
specific model (COUCHMAN; FULOP, 2006).
Existing
studying states that an AVE estimation of 0.5 or more prominent estimation is
viewed as satisfactory (BARCLAY; HIGGINS; THOMPSON, 1995). AVE of 0.5 is advocated by (HAIR et al.., 2017). Composite reliability recommended value is 0.7 (HAIR et al., 2017). If the value of AVE is low than 0.5 researchers can
still accept AVE values until 0.4 as long as composite reliability CR is >
0.6 In case of AVE = 0.4 and value of CR is > 0.6 no worry about the
convergent validity of the factor ; LARCKER, 1981; HUANG et al., 2013). Cronbach’s Alpha as recommended by Julie Pallant (2013) should be higher than 0.7. The value of spearman's
eliable rho_A should be > 0.6 (GARSON, 2009).
Table 4: Convergent
Validity-Constructs Reliability and Validity
Factors |
Cronbach's
Alpha |
rho_A |
Composite
Reliability |
Average
Variance Extracted (AVE) |
Benefit |
0.7660 |
0.7823 |
0.8409 |
0.5153 |
Loyalty |
0.8216 |
0.8266 |
0.8749 |
0.5835 |
Perceived
Ease of Use |
0.7453 |
0.7643 |
0.8381 |
0.5652 |
Perceived Efficiency |
0.7415 |
0.7532 |
0.8255 |
0.4874 |
Perceived
Functionality |
0.7524 |
0.7562 |
0.8429 |
0.5730 |
Perceived
Information Quality |
0.7665 |
0.7668 |
0.8421 |
0.5162 |
Perceived
Reliability |
0.7474 |
0.7498 |
0.8295 |
0.4933 |
Perceived
Security |
0.7660 |
0.7716 |
0.8424 |
0.5180 |
Perceived
System Quality |
0.7745 |
0.7869 |
0.8458 |
0.5239 |
Perceived
Usability |
0.8279 |
0.8309 |
0.8791 |
0.5931 |
Satisfaction |
0.7871 |
0.7876 |
0.8545 |
0.5402 |
Source:
The Researcher
Table
above is shown results of Cronbach's Alpha, rho_A, CR and AVE. The united
legitimacy is set for every one of the constructs. The delimitating factors of
united legitimacy demonstrates that the Cronbach's Alpha is greater than 0.7,
rho_A is higher than 0.6, CR is bigger than 0.7 and AVE all items are larger
than 0.5 only two factors are little bit less than 0.5, perceived reliability
is 0.49 and perceived efficiency is 0.49 there is no problem at all with these
factors because as stated above AVE value can be accepted even its = 0.4 as
long as CR > 0.6 So, all factors results are satisfactory. This was clear
enough to affirm that items speak to different or distinct hidden or latent
constructs, and thus developed their convergent validity.
Discriminant
validity, conversely identifies with whether measures that ought not to be
connected are as a rule not related. In measuring the discriminant validity,
the square root of the AVE for every factor is leveraged (FORNELL; LARCKER, 1981) as cited in (HAIR et al., 2017). Hair et al., (2017) based on Fornell and Larcker
(1981) said the square roots of AVE coefficients are then demonstrated within
the correlation matrix along its diagonal. It is observed that a squared AVE
should be more prominent than an evaluated squared correlation resulting to a
better confirmation of discriminant validity.
Table
the assessment of discriminant legitimacy was demonstrated for variables
utilized as part of the study. Table below shows square along the diagonal
underlying roots of AVE for every one of the constructs. In any case, square
foundations of developing higher AVE than off-diagonal items or coefficients in
corresponding rows and columns, henceforth, developing a proof of discriminant
legitimacy.By and large, the outcomes portrayed in tables below demonstrate
that measures for all the eleven constructs are legitimate measures of their
separate constructs in view of their factual noteworthiness and parameter
scales, following (CHOW; CHAN, 2008).
To
test discriminant validity one of the best accurate tests used is
Heterotrait-Monotrait ratio (HTMT) (HENSELER; RINGLE SARSTEDT, 2014). The values should be smaller than 1 as
noted in (ALARCÓN SÁNCHEZ DE OLAVIDE, 2015). Some researchers said values should be
less than 0.85 (KLINE, 2011) other researchers said HTMT values
should be lower than 0.90 (TEO SRIVASTAVA JIANG, 2008) as quoted in (ALARCÓN et al.,
2015).
Table 5: Discriminant Validity of Factors Square Root of
the AVE on the Diagonal (Fornell Test)
Benefit |
Loyalty |
Perceived Ease of
Use |
Perceived
Efficiency |
Perceived
Functionality |
Perceived
Information Quality |
Perceived
Reliability |
Perceived
Security |
Perceived System
Quality |
Perceived
Usability |
Satisfaction |
|
Benefit |
0.7178 |
||||||||||
Loyalty |
0.3817 |
0.7639 |
|||||||||
Perceived Ease of Use |
0.5011 |
0.1831 |
0.7518 |
||||||||
Perceived Efficiency |
0.5745 |
0.3506 |
0.5576 |
0.6981 |
|||||||
Perceived Functionality |
0.2135 |
0.1903 |
0.1371 |
0.1758 |
0.757 |
||||||
Perceived Information Quality |
0.5721 |
0.2884 |
0.6424 |
0.5721 |
0.2262 |
0.7185 |
|||||
Perceived Reliability |
0.4741 |
0.2935 |
0.3484 |
0.5055 |
0.2185 |
0.4322 |
0.7023 |
||||
Perceived Security |
0.5636 |
0.3331 |
0.5357 |
0.5513 |
0.1884 |
0.6088 |
0.4702 |
0.7197 |
|||
Perceived System Quality |
0.46 |
0.2814 |
0.6168 |
0.4705 |
0.2167 |
0.4858 |
0.3388 |
0.4504 |
0.7238 |
||
Perceived Usability |
0.5999 |
0.3281 |
0.5357 |
0.5689 |
0.1539 |
0.6243 |
0.405 |
0.6744 |
0.5451 |
0.7701 |
|
Satisfaction |
0.6254 |
0.4706 |
0.4498 |
0.561 |
0.2745 |
0.4946 |
0.4657 |
0.6459 |
0.3928 |
0.5743 |
0.735 |
Table
6: Discriminant Validity of Factors (HTMT Test)
Benefit |
Loyalty |
Perceived
Ease of Use |
Perceived
Efficiency |
Perceived
Functionality |
Perceived
Information Quality |
Perceived
Reliability |
Perceived
Security |
Perceived
System Quality |
Perceived Usability |
Satisfaction |
|
Benefit |
|||||||||||
Loyalty |
0.4672 |
||||||||||
Perceived
Ease of Use |
0.6546 |
0.2313 |
|||||||||
Perceived
Efficiency |
0.7508 |
0.4344 |
0.7489 |
||||||||
Perceived
Functionality |
0.278 |
0.2417 |
0.1881 |
0.2322 |
|||||||
Perceived Information
Quality |
0.7347 |
0.3569 |
0.8247 |
0.7427 |
0.3026 |
||||||
Perceived
Reliability |
0.6036 |
0.369 |
0.4584 |
0.6843 |
0.2802 |
0.5602 |
|||||
Perceived
Security |
0.7258 |
0.4106 |
0.703 |
0.7091 |
0.2431 |
0.799 |
0.601 |
||||
Perceived
System Quality |
0.5932 |
0.3436 |
0.8394 |
0.622 |
0.2812 |
0.6238 |
0.4457 |
0.5739 |
|||
Perceived
Usability |
0.7465 |
0.3916 |
0.6783 |
0.7091 |
0.1868 |
0.7794 |
0.5086 |
0.8489 |
0.6553 |
||
Satisfaction |
0.7829 |
0.5804 |
0.5755 |
0.7023 |
0.3537 |
0.629 |
0.5861 |
0.8243 |
0.4899 |
0.7066 |
7.4.2. Multicollinearity
A measure or
the degree of correlation among independent variables is said to be the
multicollinearity (HAIR et al., 2017). In this manner,
multicollinearity test is the progression to confirm data validity before
proceeding to regression analysis, checking multicollinearity should be
possible through bivariate of the independent variables. Each indicators
variance inflation factor (VIF) value should be less than 5 et al., 2011).
Table 7: Items VIF Values in Details
VIF |
|
Benefit1 |
1.4030 |
Benefit2 |
1.2594 |
Benefit3 |
1.5791 |
Benefit4 |
1.5318 |
Benefit5 |
1.5626 |
EFF1 |
1.2059 |
EFF2 |
1.4233 |
EFF3 |
1.4502 |
EFF4 |
1.6583 |
EFF5 |
1.2902 |
EU1 |
1.4399 |
EU2 |
1.3528 |
EU3 |
1.5335 |
Eu4 |
1.3636 |
FUN1 |
1.4555 |
FUN2 |
1.3929 |
FUN3 |
1.4966 |
FUN4 |
1.3989 |
IQ1 |
1.5661 |
IQ2 |
1.6667 |
IQ3 |
1.4706 |
IQ4 |
1.3941 |
IQ5 |
1.3045 |
Loy1 |
1.5270 |
Loy2 |
1.6198 |
Loy3 |
1.7687 |
Loy4 |
1.8319 |
Loy5 |
1.6594 |
REL1 |
1.3126 |
REL2 |
1.3914 |
REL3 |
1.6076 |
REL4 |
1.6027 |
REL5 |
1.2444 |
SQ1 |
1.3601 |
SQ2 |
1.5088 |
SQ3 |
1.4379 |
SQ4 |
1.4253 |
SQ5 |
1.4748 |
Satisf1 |
1.3872 |
Satisf2 |
1.5738 |
Satisf3 |
1.5567 |
Satisf4 |
1.5139 |
Satisf5 |
1.4631 |
Sec1 |
1.4734 |
Sec2 |
1.7553 |
Sec3 |
1.4622 |
Sec4 |
1.2726 |
Sec5 |
1.3616 |
USab1 |
1.6125 |
USab2 |
1.8450 |
USab3 |
1.8131 |
USab4 |
1.9735 |
USab5 |
1.4721 |
Source:
The Researcher
8. THE PROPOSED MODEL
In below the proposed framework of this study.
Figure 1: Ibrahim’s Proposed Framework for Systems Success Measurement
Source:
The Researcher
9. CONCLUSION
This
study aims to make a new contribution to efficiently help in systems success
measure to solve the problem of higher rate of systems fail. This study
provides a strong validated high-quality instrument so, researchers can use it
in the future in their studies. This instrument has been developed and tested
carefully. Eight PhD experts redound for validate the current instrument to
make it perfect as much as possible.
All
statistical required tests have been performed to approve the quality,
usability validity and reliability of the instrument. Results approved that the
instrument is of high quality, usability validity and reliability. This study
provided also, another perspective that can be used in the future of systems
success measurement which is ISO 25010 standard. This theory of ISO 25010 with
its factors (usability, security, efficiency, reliability and functionality)
play a vital role in the measurement of system’s success.
The
current framework and instrument have been tested in three different
universities researchers are required to test the framework and instrument in
their domains for the purpose of the generalizability and deep confirmation.
The ISO 25010 is still a new one and it’s a general metric, researchers are in
the open call to facilitate it in their studies in different domains. This
model can be used in the fields of information systems, software engineering
quality and can be used for field of software engineering testing.
Finally,
in future studies, researchers in their works are required to validate it,
whether this work is a PhD, Master or an article. The validation process is a
very good process and gives the authors a strong support for the work,
instrument and the framework. A number of the validators could be three to six
which is quite fair and enough to do guarantee for the work.
REFERENCES
ADEBOLA, B. D. (2014) The determinant of information
security practices towards organizational performance in the banking sector
evidence from Nigeria. Universiti Utara Malaysia.
AGHAZADEH, S.; PIRNEJAD, H.; ALIEV, A.; MORADKHANI, A.
(2015) Evaluating the effect of software quality characteristics on health care
quality indicators. Journal of Health Management and Informatics, v. 2,
n. 3, p. 67–73.
ALADWANI, A. M.; PALVIA, P. C. (2002) Developing and
validating an instrument for measuring user-perceived web quality. Information.;
Management, v. 39, n. 6, p. 467–476.
ALARCÓN, D.; SÁNCHEZ, J. A.; DE OLAVIDE, U. P. (2015)
Assessing convergent and discriminant validity in the ADHD-R IV rating scale:
User-written commands for Average Variance Extracted (AVE, p. Composite
Reliability (CR, p. and Heterotrait-Monotrait ratio of correlations (HTMT)
Composite Reliability (CR, p. and Heterotrait-Monotrait Ratio of Correlations
(HTMT, p. p. 1–39. In Spanish STATA Meeting
ALSHIBLY, H. H. (2015) Investigating decision support
system (DSS) success: A partial least squares structural equation modeling
approach. Journal of Business Studies Quarterly, v. 6, n. 4, p. 56.
ALVES, J. M.; WANGENHEIM, C.; LACERDA, T.; SAVARIS, A.;
WANGENHEIM, A. (2015) Adequate software quality evaluation model v1. 0. Instituto
Nacional Para Convergência Digital–INCoD, Tech. Rep.
ARDITO, C.; COSTABILE, M. F.; DE MARSICO, M.;
LANZILOTTI, R.; LEVIALDI, S.; ROSELLI, T.; ROSSANO, V. (2006) An approach to
usability evaluation of e-learning applications. Universal access in the
information society, v. 4, n. 3, p. 270–283.
ASTANI, M.; ELHINDI, M. (2008) An empirical study of
university websites. Issues in Information Systems, v. 9, n. 2, p.
460–465.
AL-AZAWEI, A.; LUNDQVIST, K. (2015) Learner differences
in perceived satisfaction of an online learning: An extension to the technology
acceptance model in an Arabic sample. Electronic Journal of E-Learning,
13(5, p. 408–426.
AL-MANASRA, E.; KHAIR, M.; ZAID, S. A.; TAHERQUTAISHAT,
F. (2013) Investigating the impact of website quality on consumers’
satisfaction in Jordanian telecommunication sector. Arab Economic and
Business Journal, v. 8, n. 1–2, p. 31–37.
BARCLAY, D.; HIGGINS, C.; THOMPSON, R. (1995) The Partial Least Squares (pls) Approach to
Casual Modeling: Personal Computer Adoption Ans Use as an Illustration.
BHARATI, P.; CHAUDHURY, A. (2004) An empirical
investigation of decision-making satisfaction in web-based decision support
systems. Decision Support Systems, v. 37, n. 2, p. 187–197.
BROWN, R. M.; MAZZAROL, T. (2006) Factors driving
student satisfaction and loyalty in Australian universities: the importance of
institutional image. In 20th Annual Australia.; New Zealand Academy of
Management (ANZAM) Conference Paper, p. 1–12.
CAVANA, R. Y.; DELAHAYE, B. L.; SEKARAN, U. (2001) Applied business research: Qualitative
and quantitative methods. John Wiley.;
Sons Australia.
CHIANG, L.; HUANG, C.-Y.; YANG, W.-C. (2011) The effect
of user’s satisfaction of web security on trust in eGovernment. In the
Proceedings of the 11th European Conference on EGovernment: Faculty of
Administration, University of Ljubljana, Ljubljana, Slovenia, p. 140.
CHIU, P.-S.; CHAO, I.-C.; KAO, C.-C.; PU, Y.-H.; HUANG,
Y.-M. (2016) Implementation and evaluation of mobile e-books in a cloud
bookcase using the information system success model. Library Hi Tech, v.
34, n. 2, p. 207–223.
CHOW, W. S.; CHAN, L. S. (2008) Social network and shared goals in organizational knowledge sharing.
CONSTANTIN, A. M. (2013) The antecedents of
e-satisfaction and e-loyalty. Timisoara Journal of Economics, v. 5, n.
2, 18, p. 236–252.
COUCHMAN, P. K.; FULOP, L. (2006) Building trust in
cross-sector R&D collaborations: Exploring the role of credible
commitments. In 22nd EGOS Colloquium, Bergen, Norway.
CRONIN, J. J.; BRADY, M. K.; HULT, G. T. M. (2000)
Assessing the effects of quality, value, and customer satisfaction on consumer
behavioral intentions in service environments. Journal of Retailing, v.
76, n. 2, p. 193–218.
DAVARPANAH, A.; MOHAMED, N. (2013) Human resource
information systems (HRIS) success factors in a public higher education
institution context. In Research and Innovation in Information Systems
(ICRIIS, p. 2013 International Conference on, p. 79–84, IEEE.
DAVIS, F. D. (1989) Perceived usefulness, perceived ease
of use, and user acceptance of information technology. MIS Quarterly, p.
319–340.
DELONE, W. H.; MCLEAN, E. R. (2016) Information systems
success measurement. foundations and trends® in Information Systems, v.
2, n. 1, p. 1–116.
DELONE, W. H.; MCLEAN, E. R. (1992) Information systems
success: the quest for the dependent variable. Information Systems Research,
v. 3, n. 1, p. 60–95.
DERNBECHER, S. (2014) Having the mind in the cloud:
organizational mindfulness and the successful use of desktop as a service. In
System Sciences (HICSS, p. 2014 47th Hawaii International Conference on, p.
2137–2146, IEEE.
DEVARAJ, S.; FAN, M.; KOHLI, R. (2002) Antecedents of
B2C channel satisfaction and preference: validating e-commerce metrics. Information
Systems Research, v. 13, n. 3, p. 316–333.
DOHI, T.; NAKAGAWA, N. (2013) Stochastic reliability and maintenance modelling. Springer.
DREHEEB, A. E.; BASIR, N.; FABIL, N. (2016) Impact of
system quality on users’ satisfaction in continuation of the use of e-learning
system. International Journal of E-Education, e-Business, e-Management
and e-Learning, v. 6, n. 1, p. 13.
DYBA, T. (2005) An empirical investigation of the key
factors for success in software process improvement. IEEE Transactions on
Software Engineering, v. 31, n. 5, p. 410–424.
EPPLER, M. J.; ALGESHEIMER, R.; DIMPFEL, M. (2003)
Quality Criteria of content-driven websites and their influence on customer
satisfaction and loyalty: An Empirical Test of an Information Quality
Framework. In IQ, p. 108–120.
FADHEL, I. E. I. (2015) An evaluation of information
system success based on students’ perspective. Universiti Utara Malaysia.
FADHEL, I. E. I.; IDRUS, S. Z. S.; IBRAHIM, A. A. E. A.;
OMAR, M.; BAHASHWAN, A. A. A.;.; AL-ANSI, A. A. M. (2018a) An integration
between information systems engineering and software engineering theories
towards engineering a novel framework of web-based systems success for
institutions based on students’ perceptions. Journal of Physics: Conference
Series, v. 1019, n. 1, p. 12081. Retrieved from http://stacks.iop.org/1742-6596/1019/i=1/a=012081
FADHEL, I. E. I.; IDRUS, S. Z. S.; IBRAHIM, A. A. E. A.;
OMAR, M.; BAHESHWAN, F.; ALBZEIRAT, M. K.;.; ALBZEIRAT, S. K. (2018b) Measuring
System Success In New Context By Adapting DM 2003 Framework With The External
Factor Management Support. Journal of Physics: Conference Series, v.
1019, n. 1, p. 12003. Retrieved from http://stacks.iop.org/1742-6596/1019/i=1/a=012003
FIELD, A. (2005) Discovering
statistics with SPSS. London: Sage.
FORSGREN, N.; DURCIKOVA, A.; CLAY, P. F.; WANG, X.
(2016) The Integrated user satisfaction model: Assessing information quality
and system quality as second-order constructs. In system
administration. Communications of the Association for Information Systems,
v. 38, n. 1, p. 39.
FORNELL, C.; LARCKER, D. F. (1981) Evaluating structural
equation models with unobservable variables and measurement error. Journal
of Marketing Research, p. 39–50.
GORLA, N.; SOMERS, T. M.; WONG, B. (2010) Organizational
impact of system quality, information quality, and service quality. The
Journal of Strategic Information Systems, v. 19, n. 3, p. 207–228.
GUADAGNOLI, E.; VELICER, W. F. (1988) Relation to sample
size to the stability of component patterns. Psychological Bulletin, v.
103, n. 2, p. 265.
HAIR JR, J. F.; HULT, G. T. M.; RINGLE, C.; SARSTEDT, M.
(2017) A primer on partial least squares structural equation modeling (PLS-SEM)
Thousand Oaks: Sage Publications.
HENSELER, J.; RINGLE, C. M.; SARSTEDT, M. (2014) A new
criterion for assessing discriminant validity in variance-based structural
equation modeling. Journal of the Academy of Marketing Science, v. 43,
n. 1, p. 115–135.
HOLBROOK, A. L.; KROSNICK, J. A.; MOORE, D.; TOURANGEAU,
R. (2007) Response order effects in dichotomous categorical questions presented
orally: The impact of question and respondent attributes. Public Opinion
Quarterly, v. 71, n. 3, p. 325–348.
HUANG, C.-C.; WANG, Y.-M.; WU, T.-W.; WANG, P.-A. (2013)
An empirical analysis of the antecedents and performance consequences of using
the moodle platform. International Journal of Information and Education
Technology, v. 3, n. 2, p. 217.
JANSEN, H.; HAK, T. (2005) The productivity of the
three-step test-interview (TSTI) compared to an expert review of a
self-administered questionnaire on alcohol consumption. Journal of Official
Statistics, v. 21, n. 1, p. 103.
JEON, M. M. (2009) Impact of perceived website service
quality on customer e-loyalty on a lodging website. Iowa State University.
JONGCHUL, O.; YOON, S.-J. (2014) Validation of haptic
enabling technology acceptance model (HE-TAM): Integration of IDT and TAM. Telematics
and Informatics, v. 31, n. 4, p. 585–596.
KHAWAJA, K. F.; BOKHARI, R. H. (2010) Exploring the
factors associated with quality of website. Global Journal of Computer
Science and Technology, v. 10, n. 14, p. 37–45.
KHRED, D. A. (2017) Systems
development in our country.
KIRAN, K.; DILJIT, S. (2011) Antecedents of customer
loyalty: Does service quality suffice? Malaysian Journal of Library.;
Information Science, v. 16, n. 2, p. 95–113.
KLINE, R. B. (2011) Principles
and practice of structural equation modeling, 3rd edn Guilford Press. New York.
LIAW, S.-S.; HUANG, H.-M. (2013) Perceived satisfaction,
perceived usefulness and interactive learning environments as predictors to
self-regulation in e-learning environments. Computers.; Education, v.
60, n. 1, p. 14–24.
LIU, I.-F.; CHEN, M. C.; SUN, Y. S.; WIBLE, D.; KUO,
C.-H. (2010) Extending the TAM model to explore the factors that affect
Intention to Use an Online Learning Community. Computers.; Education, v.
54, n. 2, p. 600–610.
LUDIN, I. H. B. H.; CHENG, B. L. (2014) Factors
influencing customer satisfaction and e-loyalty: online shopping environment
among the young adults. Management Dynamics in the Knowledge Economy, v.
2, n. 3, p. 462.
MACCALLUM, R. C WIDAMAN, K. F.; PREACHER, K. J.; HONG,
S. (2001) Sample size in factor analysis: The role of model error. Multivariate
Behavioral Research, v. 36, n. 4, p. 611–637.
MADAN, A.; DUBEY, S. K. (2012) Usability evaluation
methods: a literature review. International Journal of Engineering Science
and Technology, v. 4, n. 2, p. 590–599.
MALIK, B. H.; SHUQIN, C.; MASTOI, A. G.; GUL, N.; GUL,
H. (2016) Evaluating citizen e-satisfaction from e-government services: A Case
of Pakistan. European Scientific Journal, ESJ, v. 12, n. 5.
MARDIANA, S.; TJAKRAATMADJA, J. H.; APRIANINGSIH, A.
(2015) Validating the Conceptual Model for Predicting Intention to Use as Part
of Information System Success Model: The Case of an Indonesian Government
Agency. Procedia Computer Science, n. 72, p. 353–360.
MBIWA, P. N. (2014) Evaluating
the quality of management information systems: A case of SACCOs in Nairobi,
Kenya. University of Nairobi.
MCGILL, T. J.; HOBBS, V. J.; KLOBAS, J. E. (2003) User
developed applications and information systems success: A test of DeLone and
McLean’s model. Information Resources Management Journal, v. 16, n. 1,
p. 24–45.
MEBRATE, T. (2010) A
framework for evaluating academic website’s quality from students’ perspective.
TU Delft, Delft University of
Technology.
MOHAMMADI, H. (2015) Investigating users’ perspectives
on e-learning: an integration of TAM and IS success model. Computers in
Human Behavior, n. 45, p. 359–374.
OFORI, K. S.; LARBI-SIAW, O.; FIANU, E.; GLADJAH, R. E.;
BOATENG, E. O. Y. (2016) Factors influencing the continuance use of mobile
social media: The effect of privacy concerns. Journal of Cyber Security and
Mobility, v. 4, n. 3, p. 105–124.
OLSON, K. (2010) An examination of questionnaire
evaluation by expert reviewers. Field Methods, v. 22, n. 4, p. 295–318.
PADAYACHEE, I.; KOTZE, P.; VAN DER MERWE, A. (2010) ISO
9126 external systems quality characteristics, sub-characteristics and domain
specific criteria for evaluating e-Learning systems. The Southern African
Computer Lecturers’ Association, University of Pretoria, South Africa.
PALLANT, J. (2007) SPSS
survival manual: A step guide to data analysis using SPSS for Windows (3rd
Ed) Australia: Allan.; Unwin.
PALLANT, J. (2013) SPSS survival manual. McGraw-Hill
Education (UK)
PAPANIKOLAOU, K.; MAVROMOUSTAKOS, S. (2008) Web2Train: A
design model for corporate e-learning systems. In BIS (Workshops), p.
155–163.
PARK, N.; RHOADS, M.; HOU, J.; LEE, K. M. (2014)
Understanding the acceptance of teleconferencing systems among employees: An
extension of the technology acceptance model. Computers in Human Behavior,
n. 39, p. 118–127.
PRESSER, S.; BLAIR, J. (1994) Survey pretesting: Do
different methods produce different results? Sociological Methodology,
p. 73–104.
ROCHA, Á. (2012) Framework for a global quality
evaluation of a website. Online Information Review, v. 36, n. 3, p.
374–382.
SEKARAN, U. (2003) Research
methods for business: A skill building approach. John Wiley.; Sons.
SEKARAN, U.; BOUGIE, R. (2010) Research methods for business: A skill building approach. Chichester: John Willey.; Sons Ltd. 2010.
SELVAKUMAR, J. J. (2016) Impact of service quality on
customer satisfaction in public sector and private sector banks. Purushartha: A
Journal of Management Ethics and Spirituality, v. 8, n. 1.
SENATE, A. (2001) Universities
in crisis. Canberra:
Commonwealth of Australia.
SHIRATUDDIN, N. (2015) Evaluation of e-Book applications
using ISO 25010. In Technology Management and Emerging Technologies (ISTMET,
p. 2015 International Symposium on, p. 114–118, IEEE.
STEVENS, J. (1992) Applied
multivariate statistics for the social sciences (2nd edition) Hillsdale, NJ:Erlbaum.
TABACHNICK, B. G.; FIDELL, L. S. (2007) Using multivariate statistics. Allyn.; Bacon/Pearson Education.
TANDON, U.; KIRAN, R.; SAH, A. (2017a) Analyzing
customer satisfaction: users perspective towards online shopping. Nankai
Business Review International, v. 8, n. 3, p. 266–288.
THEIS, B.; FROOD, J.; NISHRI, D.; MARRETT, L. D. (2002)
Evaluation of a risk factor survey with three assessment methods. Chronic
Diseases and Injuries in Canada, v. 23, n. 1, p. 1.
TEO, T. S. H.; SRIVASTAVA, S. C.; JIANG, L. (2008) Trust
and electronic government success: An empirical study. Journal of Management
Information Systems, v. 25, n. 3, p. 99–132.
UNIT, M. C. AND B. S. (2013) What thresholds should I use for factor loading cut-offs? Retrieved
from http://imaging.mrc-cbu.cam.ac.uk/statswiki/FAQ/thresholds
VAEZI, R.; MILLS, A.; CHIN, W.; ZAFAR, H. (2016) User
satisfaction research in information systems: Historical roots and approaches. Communications
of the Association for Information Systems, v. 38, n. 27, p. 501–532.
VALVI, A. C.; WEST, D. C. (2013) E-loyalty is not all
about trust, price also matters: extending expectation-confirmation theory in
bookselling websites. Journal of Electronic Commerce Research, v. 14, n.
1, p. 99.
VAN IWAARDEN, J.; VAN DER WIELE, T.; BALL, L.; MILLEN,
R. (2004) Perceptions about the quality of web sites: a survey amongst students
at Northeastern University and Erasmus University. Information.; Management,
v. 41, n. 8, p. 947–959.
WANG, Y.-S.; LIAO, Y.-W. (2008) Assessing eGovernment
systems success: A validation of the DeLone and McLean model of information
systems success. Government Information Quarterly, v. 25, n. 4, p.
717–733.
WIXOM, B. H.; WATSON, H. J. (2001) An empirical
investigation of the factors affecting data warehousing success. MIS
Quarterly, p. 17–41.
WOLFINBARGER, M.; GILLY, M. C. (2003) eTailQ:
dimensionalzing, measuring and predicting etail quality. Journal of
Retailing, v. 79, n. 3, p. 183–198.
ZAIED, A. N. H. (2012) An integrated success model for
evaluating information system in public sectors. Journal of Emerging Trends
in Computing and Information Sciences, v. 3, n. 6, p. 814–825.
ZEHIR, C.; SEHITOGLU, Y.; NARCIKARA, E.; ZEHIR, S.
(2014) ES-quality, perceived value and loyalty intentions relationships in
internet retailers. Procedia-Social and Behavioral Sciences, n. 150, p.
1071–1079.
ZEITHAML, V. A.; BERRY, L. L.; PARASURAMAN, A. (1996)
The behavioral consequences of service quality. The Journal of Marketing,
p. 31–46.