Adaptation and empirical evaluation of the questionnaire on students ’ motivation towards science learning

The aim of this study is to verify the psychometric properties of the translation of the questionnaire which is designed for self-assessment of students’ motivation toward science learning, SMTSL. Besides being translated and adapted to Serbian language, the questionnaire was adapted to the specific properties of chemistry as a school subject. The administered questionnaire consisted of 29 items in the five-point Likert scale and contained five subscales which assessed a sense of self-efficacy for learning chemistry, active learning strategies, chemistry learning value, performance goal and achievement goal. The suitability of the theoretical model and psychometric characteristics of the questionnaire were assessed on the sample of 741 grammar school students. The results show that the tested model has good fit indicators. The calculated values of the indicators of reliability and representativeness indicate quite satisfactory psychometric properties of the questionnaire and it can be used in further research.

A significant amount of studies, which are based on the increasingly complex interrelations of various models of motivational variables, is focused on correlation between students' motivation for learning and academic achievement (Covington, 2000;Pintrich, 2000).There are a growing number of research studies aimed at studying students' motivation for learning specific subjects, which are based on the idea that students can express different motivational tendencies depending on the field (Lee & Brophy, 1996).However, the findings that explicate the relation of variables covered by modern motivation and achievement theories in specific academic areas are still relatively few.
Until now, few studies have focused on different factors that can play an important role in the development of student motivation to learn chemistry teaching contents.Zusho, Pintrich, and Coppola (2003) provided evidences that the sense of self-efficacy and the assessment of the value of the assignment are predictors of achievement in chemistry.In their study, Brković, Petrović-Bjeković, and Zlatić (1998) have found that secondary school students are the least motivated to learn chemistry from all science subjects.This negative trend which is still present in schools can be explained by the fact that there are numerous difficulties in studying chemistry teaching contents.Previous studies in the field of chemical education have shown that problems arise from the use of chemical language which is unfamiliar and abstruse to students (Halkia & Mantzouridis, 2005).Additional difficulties in learning chemistry are directly related to the specific nature of chemistry contents, which covers two levels of formal concepts (macroscopic and submicroscopic level) that should be brought in mutual connection, as well as in connection with the observed phenomena, and which should be communicated not only by using a specialized chemical terminology, but also through various forms of symbolic representation (Taber, 2013).
The increased interest of researchers for studying academic motivation to learn natural sciences inspired the development of several instruments for measuring student motivation based on different theoretical perspectives (Glynn & Koballa, 2006;Tuan, Chin, & Shyang, 2005;Velayutham, Aldridge, & Fraser, 2011).According to our knowledge, despite their existence there is still no instrument intended to measure student motivation to learn chemistry in Serbian language.In this context, the purpose of this paper is to evaluate the questionnaire for measuring the motivation for learning natural sciences developed by Tuan et al. (2005), which is characterized by clear theoretical foundations, good psychometric characteristics, and ease of use.In this paper, the questionnaire is adapted to the specifics of chemistry due to the fact that in Serbian curriculum the integrated contents of natural sciences is studied only in the first cycle of primary education and upbringing.In the second cycle of primary education, as well as in secondary school, the contents of natural sciences are clearly differentiated into separate subject matters: biology, physics, and chemistry.
The SMTSL (Student's Motivation Toward Science Learning, Tuan et al. (2005) questionnaire defines motivation as a multidimensional construct that includes: 1. Self-efficacy -the level of students' confidence in their abilities of performing assignments successfully.2. Active learning strategies -students have active role in using different strategies when adopting new materials based on understanding and connecting with knowledge already acquired.
3. Science learning value -learning natural science enables students to acquire competence in solving problems, develop critical thinking and understand the importance of science in everyday life.If students are in situation to perceive these values of learning the content of natural sciences, they will be motivated to learn.4. Performance goal -in the process of learning materials of natural sciences students aim to compete with other students and draw teacher's attention.5. Achievement goal -in the process of learning science students feel satisfaction as they develop their own competence and achieve success.6. Learning environment stimulation -the environment such as the curriculum, the teacher presenting the lecture and student interactions are all affecting student motivation for learning.
The perceived self-efficacy is the core concept of the socio-cognitive theory of motivation created by Bandura (1997;2001).Bandura defines the perceived self-efficacy as people's judgments of their capabilities to organize and execute the actions necessary to achieve the desired goals (Bandura, 1997).Instead of the actual skills owned by the individual, estimates of self-efficacy are related to what a person can do regardless of his/her actual skills.The socio-cognitive theory postulates the existence of two types of expectations: expectation regarding the outcome and expectations regarding personal efficacy.Expectation regarding the outcome refers to the individual's belief that a specific behaviour will lead to a desired goal, while expectation regarding self-efficacy refers to the belief about what a person can do to implement actions the ultimate goal of which is achieving the expected outcome.The socio-cognitive theory has been used to explain the behaviour of individuals in many areas, including education (Bandura, 1997).
The individual's self-efficacy beliefs affect the cognitive, motivational and affective processes, and it is a strong determinant of defined objectives, outcome expectations, choice of activities and willingness to invest effort (Bandura, 1997;2001).The students' lack of awareness of their own cognitive capacities can confine their functioning in situations that require identifying and using new learning strategies (Mirkov, 2005).
Previous studies have identified two important variables of academic motivation: the students' personal interests and goals (Hidi & Harackiewicz, 2000).Achievement goal theory is not focused on the question of what students want to achieve but why.Why, for example, would students like to get the highest grade?According to this theoretical framework there are two possible reasons: (1) they want to learn and understand the material, and (2) they just want to show others (teachers, peers, parents) that they are more successful than the others (Maehr & Zusho, 2009).In this sense, Ames (1992) distinguishes two broad motivational orientations or patterns: the mastery goals orientation and performance goals orientation.While performance-oriented students try to outperform the others and accept assignments for which they are confident that they will do them successfully, mastery-oriented students choose challenging tasks and evaluate their own achievement compared to their previous results, rather than results of others (Maehr & Zusho, 2009).
The students' feelings toward school, the use of learning strategies, the experience of efficacy, affective reactions to success and failure are related to their motivational orientation (Mirkov, 2008).Mastery-oriented students believe that the main goal of schooling is to master the assignment and achieve competences in what is taught in school, improve the knowledge, and understand the materials being presented (Mirkov, 2008;Vizek-Vidović, Vlahović-Štetić, Rijavec, & Miljković, 2003).In this case, the learning material is believed to be the goal in itself (Mirkov, 2008).Performance-oriented students are unconcerned about how to understand the material and relate it to the already acquired knowledge: their objective is to outperform their peers.In achieving this objective they apply superficial and short-term learning strategies (Vízek-Vidović et al., 2003).In this case, success is primarily determined by comparing the own achievements with those of the others (Maehr & Zusho, 2009).For these students, learning the materials is a means to accomplish the objective (Mirkov, 2008).In both motivational orientations competence plays a significant role.In the case of mastery orientation, competence is self-evaluated against the personal standards of achievement.Achievement or performance-oriented students, on the other hand, usually see competence as a property of a privileged few who are more capable than most of the others (Maehr & Zusho, 2009).
Students who are motivated to learn trigger cognitive and metacognitive strategies that enable them to apply scientific insights to understand the world around them (Lee & Brophy, 1996).By applying active learning strategies students become agents in the acquisition and application of new knowledge (Tuan et al., 2005).Active learning is an intrinsically motivated activity as it satisfies the deep psychological need for a sense of competence and autonomy (Deci & Ryan, 2000).On the other hand, students who are not intrinsically motivated will be using strategies that enables them to meet their expectations with the least possible effort invested (Lee & Brophy, 1996).
According to the expectancy-value model (Eccles & Wigfield, 2002), the perception of value of learning directly depends on the characteristics of various tasks and their influence on student motivation.In this sense, the estimated value of the assignment is subjective because different students ascribe different values to the same learning activity.The evaluation of meaningfulness of the task is determined by how it is related to the student's current and future goals.The assignment can be valuable for the students as it allows them to achieve important goals, even if they are not interested in the assignment itself.For example, students may invest an effort in order to satisfy their parents, show-off in front of their peers, while still dislike the learning process.Although it is obvious that such evaluation of assignment is more related to extrinsic motivation, it can also be directly associated with internalized short-term and long-term goals (Eccles & Wigfield, 2002).
Bearing in mind the above theoretical assumptions, the authors believe that the theoretical foundations of the underlying questionnaire are good.To the authors' knowledge, the SMTSL questionnaire has seen two adaptations so far: in Turkish (Yilmaz & Cavas, 2007) and Greek (Dermitzaki, Stavroussi, Vavougios, & Kotsis, 2013).Validation of the Turkish translation of the questionnaire (Yilmaz & Cavas, 2007) has been conducted on a sample of 659 elementary school pupils (sixth, seventh and eighth grade).As indicated by the data, the psychometric properties of the instrument were satisfactory.However, it should be noted that the adaptation eliminated two items (15 and 21) from the original scale because the results of the exploratory factor analysis have indicated low factor saturation along these items.After this adaptation the questionnaire has been used in a number of other studies which assessed students' motivation to learn natural sciences (Cavas, 2011;Güvercin, Tekkaya, & Sungur, 2010;Sevinc, Ozmen, & Yigit, 2011).
The Greek adaptation (Dermitzaki et al., 2013) has been validated on a sample of 350 students from two universities in Greece, prospective teachers of physics.The obtained results have confirmed the adequacy of the six-factor theoretical model of academic motivation, which was proposed by the authors of the SMTSL.It is particularly important to emphasize that psychometric properties of the questionnaire were found satisfactory when applied to a sample of students in different cultural environment.
Findings of previous research studies conducted with the aim of adapting and validating the SMTSL questionnaire (Dermitzaki et al., 2013;Tuan et al., 2005;Yilmaz & Cavas, 2007) have shown that its psychometric properties are good.Accordingly, the aim of this paper is to evaluate the Student's Motivation Toward Science Learning questionnaire (Tuan et al., 2005), that is its version that has been translated to our language and adapted to our culture, as well as to the specifics of chemistry as a teaching subject.The results of psychometric validation of the scale on a Serbian sample could open the way for its use in future studies.

Method Sample
The study involved 741 students from grammar school of general studies on the territory of Vojvodina, age 15-18.A sample is of convenient type, consisting of first (35.8%),third (31.8%) and fourth (32.4%) grade students from grammar school in Novi Sad, Sremska Mitrovica, Stara Pazova and Bečej, of which 40.5% were males, 58.7% females, while 0.8% of respondents remained undeclared.In all the schools, the consent of the school principal and the chemistry teachers had been obtained for conducting the research.When considering students' school achievement 50.6% finished the previous school year with excellent grade, 33.1% with very good grade, 8.6% with good grade, and 0.8% of students finished the school year with sufficient grade.
The questionnaires were submitted to the students at the beginning of the 2013/2014 school year, and students were informed that the research was anonymous, that they were allowed to withdraw, and that there would be no negative consequences as a result of unwillingness to complete the questionnaire.Respondents did not report any problems when filling the research instrument which lasted about 20 minutes.

Instrument and procedure
The research was based on the SMTSL questionnaire (Student's Motivation Toward Science Learning), constructed by a group of authors (Tuan et al., 2005), which was adapted for the purpose of this research by the authors of this paper.Respondents answered by circling the number on a five-point scale that ranged from completely agree (coded five); to completely disagree (coded one).
Adapting the instrument for this study consisted of several phases.Items were first translated using the double translation method and then adapted to Serbian language, whereas items were formulated to relate to students' motivation for learning chemistry.The translation process involved three university teachers whose native language is Serbian.In the first step, each teacher individually translated the questionnaire from English into Serbian, after which the translations were harmonized.This was followed by a preliminary survey on a sample of 136 grammar school students.The results of this survey have shown that the coefficients of internal consistency (Cronbach's alpha) of subscales were .71,.79,.80,.58i .80,respectively.These values could be considered acceptable, except for the subscale related to Learning environment stimulation (α = .22).This finding required additional analysis.Certain deficiencies and ambiguities were identified while speaking with the school psychologist, the professor of chemistry and three students.In addition, some minor changes in wording of some items were introduced and the number of items reduced.Namely, from the set of 35 statements contained in the original questionnaire 29 items were chosen for which the authors of the questionnaire believed to relate to five aspects of motivation for learning: self-efficacy, active learning, and appreciation of chemistry, achievement orientation and learning orientation.The retained items were subjected to confirmatory factor analysis in order to verify the adequacy of theoretical distribution of items along the scales of the SMTSL inventory, and then the descriptive and psychometric indicators were calculated.

Data processing
The descriptive and psychometric indicators of the questionnaire were calculated by using the IBM SPSS (version 21) software package, whereas the confirmatory factor analysis was conducted using the Lavaan software package, written for the R environment (Rosseel, 2012).Given the deviation of multivariate distribution from the normal (Mardia's normalized multivariate kurtosis coefficient was 92.26), the model was estimated using the weighted least-squares method (WLSM, Chu, Kalaba, & Spingarn, 1979).The model's compliance or goodness-of-fit was assessed using the following indices: the chi-square (χ 2 ), the chi-square/ degrees of freedom ration (χ 2 /df) and CFI group of the so-called incremental fit indices that compare the model with the zero model, i.e. the model in which covariances are zero, as well as RMSEA and SRMR.

Results
Factor structure of the questionnaire Confirmatory factor analysis was aimed at verifying the adequacy of theoretical distribution of items along the subscales of the SMTSL questionnaire in Serbian language.The tested model assumes the existence of five variables in the basis of the questionnaire applied: self-efficacy (SE), active learning strategies (AL), chemistry learning value (CL), performance goal (PG) and achievement goal (AG).Each of these five latent variables in the model was manifested by items that build the appropriate SMTSL scale.The obtained indicators of fit of the received data into the model are presented in Table 1.Based on data presented in Table 1, indicators of fit of the model with five subscales (SE, AL, CL, PG, AG) are fine.Thus, taking into account a more liberal criterion for a sample larger than 200 respondents, according to which values of χ 2 /df ≤ 5 (Marsh & Hocevar, 1985) indicate a good suitability, the obtained values show that the empirical data are consistent with the assumed model.The results show that the root mean square error of approximation (RMSEA) is .072(.068 -.076), indicating reasonable errors of approximation in the population (Hoe, 2008;Kline, 2011;MacCallum, Browne, & Sugawara, 1996).The standardized root mean square residuals (SRMR) were .06,indicating a good fit (SRMR ≤ .07,Bagozzi, 2010).Comparative fit index (CFI) was .90,indicating a acceptable fit index (Chirkov, Ryan, Kim, & Kaplan, 2003;Greene, Miller, Crowson, Duke, & Akey, 2004;Hoe, 2008;Kodžopeljić, Dinić, & Čolović, 2014;Marsh et al., 2010).Based on the results it can be concluded that the model's fit indices were satisfactory.Figure 1 shows the standardized factor loadings for the five-factor model.As shown in Figure 1 can be seen, all items except item 21 have high loads, suggesting that they represent good measures for their parent factors.However, weak factor saturation can be observed for item number 21.This is certainly a question that needs to be further addressed when checking other indicators of metric properties of the questionnaire.

Descriptive indicators of the questionnaire scales
Many statistical techniques are based on the assumption that the distribution of the results of the dependent variable is normal.Therefore, the normality of distributions obtained in this study has also been checked.Table 2 shows the descriptive statistical indicators of summary scores along the subscales of the questionnaire.Based on the indicators of asymmetry it can be seen that the distribution is shifted towards higher values along all subscales.The reason for this distribution shift should be sought in the structure of the sample, which consists of grammar school students, whose motivation to learn chemistry is higher than that of secondary school students.Although values of the Kolmogorov-Smirnov test are statistically significant for all subscales of the questionnaire, the values of skewness and kurtosis are acceptable.Taking values of skewness and kurtosis as the criterion which, according to some authors (Tabachnick & Fidell, 2007) should be in the range of -1 to +1, then the hypothesis of normality of distribution is confirmed along all scales, except for the subscale of learning orientation.However, the assumption of normality of distribution can be satisfied using the method of data normalization (by transforming the variable).In this case, applying the appropriate transformation (mapping and square root) satisfactory parameters was obtained (skewness .30,kurtosis -.60).According to the other, less stringent criterion, values of skewness and kurtosis should not exceed the standardized values of 2 and 7, respectively (Finney & DiStefano, 2006).Thus, according to both criteria, all subscales have acceptable values of skewness and kurtosis, which confirms the assumption of normal distribution of the data obtained.

Psychometric properties of the questionnaire
Representativeness of items expressed with the normalized Kaiser-Meyer-Olkin coefficient (KMO) is high and is .87(the lower limit is .50,Field, 2009).Values of the Cronbach's alpha coefficient of subscales are calculated for simple summation of items for each scale.Table 3 offers a comparative overview of obtained values of internal consistency and the previous two studies.It should be noted that in the studies conducted in Greece (Dermitzaki et al., 2013) and Turkey (Yilmaz & Cavas, 2007) psychometric characteristics were determined based on a 35-item instrument, similar to the original questionnaire.However, after the analysis, the authors of both studies suggested to eliminate two items from each study, so that studies (Cavas, 2011) were conducted using a 33-item revised version.The values of the Cronbach's alpha coefficient of reliability lower than .70 are considered unacceptable (Fajgelj & Janičić, 2008).As indicated by the results, all subscales have good reliability and representativeness.However, it should be noted that value of the Cronbach's alpha coefficient for the PG subscale is lower than that in the original questionnaire (Tuan et al., 2005).
The results of item-total analysis for all subscales of the questionnaire are shown in Table 4. Taking .30(Nunnally & Bernstein, 1994) as the lower limit of acceptability for the correlation of items with the total score on the scale (corrected item-total correlations), it can be seen that this criterion is almost fully satisfied.Namely, excluded item 21, whose correlation with the total score on the scale is .11, the correlation varies between .34 and .66,which indicates acceptable item discriminativeness.

Discussion
One of the issues drawing science and research attention in recent years is the multidimensional nature of motivation and its dependence of the characteristics of the context.However, the number of studies focusing on the specifics of students' motivation to learn natural sciences is still small, especially in Serbian language.In this context, there is a clear need for a reliable, valid and contextually adapted instruments intended for studying academic motivation of students.The main objective of this study was empirical evaluation of the Serbian version of the SMTSL questionnaire.
The SMTSL questionnaire was initially designed as a self-assessment scale consisting of 35 items, and six subscales, intended for assessing students' motivation for learning integrated natural sciences.The questionnaire used in this study consists of 29 items and is specifically modified for the purpose of evaluating motivation to learn chemistry.Namely, after preliminary research, the subscale that estimates the learning environment stimulation was found to have unacceptable value of internal consistency which is significantly different from the value provided by the authors of the questionnaire (Tuan et al., 2005) and the results obtained in other studies (Dermitzaki et al., 2013;Yilmaz & Cavas, 2007).The reason for this deviation from the results of other studies may be found in the formulation of questions.Formulations of items such as, for example, "I am willing to participate in chemistry course because the content is exciting and changeable" and the like, do not reflect, at least not fully, the characteristics of the educational system in Serbia.Namely, students are "not in the situation to choose the school, the teachers, and the teaching content" (Bosanac, 2011: 97); thus, statements defined in this way may lead them to confusion.Chemistry, for example, is a compulsory subject in all four years of grammar school of general studies, and students do not have a choice.This conclusion was derived after speaking with the school psychologist, the chemistry teacher and three students who have pointed out these ambiguities.In short, from a set of 35 items of the original questionnaire, the Serbian version whose psychometric properties were checked for retained 29 items divided into five subscales: perceived self-efficacy, the use of active learning strategies, appreciation of chemistry, achievement and learning oriented motivation.
It is necessary to point out the theoretically established high correlations between subscales of the SMTSL questionnaire in the Serbian version.Specifically, according to the results, there is a high correlation between the students' perceived self-efficacy, the application of active learning strategies, appreciating the importance of chemistry as a science and motivational orientation.According to the socio-cognitive theory of motivation (Bandura, 1997;2001), students who are confident in their academic capabilities monitor their work time more effectively and show more persistence than do equally able peers with low self-efficacy.This means that the students' sense of self-efficacy is associated with the application of learning strategies (Mirkov, 2008).In this sense, we can say that it is an empirically confirmed assumption that students with high sense of self-efficacy will act as agents in the process of learning.On the other hand, when confronted with assignments which they perceive as valuable and meaningful, students will apply different strategies and invest more intellectual effort in order to reach their goals (Eccles & Wigfield, 2002;Pintrich & Schunk, 1996).
The results also support the assumption that the sense of self-efficacy is associated with the students' motivational orientation (Mirkov, 2008) for which it can be said that directs the behaviour of students in learning (Elliot & Murayama, 2008).According to some authors (Wigfield & Cambria, 2010), there is a clear connection between individual needs and goals with the development of competencies in the learning process.The extent to which the students will participate in the learning process depends on whether their motivation is learning or achievement oriented (Ames, 1992).Students focused on acquiring competencies, deepening knowledge and understanding the material (i.e.their motivation is learning oriented) will be involved to a greater extent and apply a variety of strategies, investing thereby more intellectual effort in order to achieve their goals (Pintrich & Schunk, 1996).
Based on the fact that in current studies in various fields of psychology structural equation modelling is a typical element of the research inventory (Lazarevic, 2008), the paper had evaluated the suitability of the theoretical model proposed by the authors of the original version of the SMTSL questionnaire (Tuan et al., 2005) with the obtained empirical data.By using confirmatory factor analysis, the proposed structure of the questionnaire has been confirmed.Specifically, the results obtained by the authors of the original SMTSL questionnaire (Tuan et al., 2005) in their research are comparable to those obtained in this study.The results have shown that the five-factor model of motivation consisting of factors of perceived self-efficacy, application of active learning strategies, appreciation of chemistry, achievement and learning oriented motivation is fully in line with the initial hypothesis about the structure of the questionnaire set by its authors (Tuan et al., 2005), as well as with findings of other foreign empirical verifications of the SMTSL questionnaire (Dermitzaki at al., 2013;Yilmaz & Cavas, 2007).
When comparing the values of the Cronbach's alphas obtained by the authors of the original SMTSL questionnaire (Tuan et al., 2005) with the values obtained in other studies (Dermitzaki et al., 2013;Yilmaz & Cavas, 2007) and those obtained in this study, it is clear that the questionnaire has a good reliability.Compared to the original questionnaire (Tuan et al., 2005), the only novelty is the lower value of the reliability coefficient for the subscale which assesses students' motivational orientation to achievement.However, taking into account the small number of the items on this scale, this value can be considered acceptable (Fajgelj & Janičić, 2008).A similar value for the reliability coefficient was also obtained in studies of other authors (Yilmaz & Cavas, 2007).
Item 21 ("I participate in chemistry course to get a good grade") deserves special attention.The results obtained by the confirmatory factor analysis indicated a weak saturation factor for this item.In addition, with the exclusion of this item, the reliability coefficient along the subscale of achievement orientation will increase from .57to .72.Also, the coefficient of discriminativeness of this item has not proved satisfactory.Taking all the above into consideration, as a possible solution, we propose excluding item 21 from the Serbian version of the questionnaire.This solution is also suggested by the authors of other research studies related to the psychometric verification of the SMTSL questionnaire on specific national samples (Dermitzaki et al., 2013;Yilmaz & Cavas, 2007).

Conclusion
The paper presents the results of study aimed at empirical evaluating the psychometric properties of Serbian translation of the SMTSL questionnaire.Based on the results of this study it can be concluded that the questionnaire has good psychometric properties, as also confirmed in several previous research studies.Based on all the above, it can be concluded that the adapted and translated Serbian version of the SMTSL questionnaire is reliable and representative, and has enough quality for using it in research and practical purposes.In addition to good psychometric characteristics, the SMTSL scale is characterized by simple application, ability to adapt to other subject matters, and relevance in different cultural environments.
Considerable limitation of the research presented in this paper is related to the fact that the sample is composed exclusively of grammar school students.Such a sample structure evidently reduces the possibility to make general conclusions after the obtained results.In this sense, it will be useful for future research to include students attending other types of secondary schools in Serbia.

Table 1
Goodness-of-Fit Statistics for Model of the SMTSL Note. ** p<.01; χ 2 -chi-square values; RMSEA -Root mean square error of approximation; SRMR -The standardized root mean square residuals; CFI -Comparative fit index

Table 3
Reliabilites of the SMTSL

Table 4
Characteristics of individual items