There is still potential for future development and innovation in STEAM education in Indonesia. For example, based on a systematic literature review conducted by (Farwati et al., 2021), the implementation of this educational approach has not been widely distributed as it mainly concentrates on West and East Java. The findings from two earlier studies contradict the Indonesian government’s efforts to enhance digitalization in education and achieve equitable education in Indonesia (Zamjani et al., 2020). Furthermore, in our previous study, we found that the development of STEM/STEAM education in Indonesia mainly emphasizes hands-on activities with the limited utilization of technologies, while STEAM education is relatively new (Laksmiwati et al., 2023). Addressing this imperative, we were intrigued to investigate how to assist teachers in successfully bringing STEAM education into their classroom practices as well as to support the Indonesian government’s endeavor to utilize digital technologies.

In our previous study, we developed instructional design heuristics to support teachers in designing STEAM lessons in a hybrid learning environment (Laksmiwati et al., 2024). In the study, the developed instructional design heuristics have been manifested into 13 design principles, which are expected to be used in a broader range of STEAM implementation. Therefore, in this manuscript, we report our investigation of the quality of the design principles. This initiative seeks to forge the contribution to future explorations in implementing STEAM education in Indonesia. Hence, the main objective of this study was to propose main dimensions which significantly support future research to empower the implementation of STEAM education in Indonesian classrooms. STEAM is an instructional approach that combine five different disciplines namely Science, Technology, Engineering, Art and Mathematics (Syahmani et al., 2021). STEM can empower individuals with the skills to succeed and adapt to new work environments (Setyaningsih et al., 2022).

The focal point of this study was to explore the main dimensions that contribute to further investigation. In this study, we focused on investigating teacher experts' evaluation of the quality of our proposed design principles. By examining teacher experts' evaluation, we aimed to contribute to the development of main dimensions in order to implement STEAM in mathematics classrooms, in particular in Indonesia. Hence, the following question guided our study: What are the teacher experts' recommendations on the proposed design principles and future explorations.

In this section, we present the way in which the thirteen design principles used in this study relate to existing literature. Therefore, in the subsequent section, we discuss literature studies and provide a comprehensive review of relevant literature to each design principle employed in our work.

According to (Arık & Topçu, 2022), there is an open-ended nature of the design process activities in STEAM learning with a focus on the engineering design process. Furthermore, (Syahmani et al., 2021) on the nature of the engineering design process indicates that context could influence how students employ design practices. Hence, it is crucial to contemplate that teachers will be present in learning activities to bolster students' learning adequately. Based on (Harju-Luukkainen & Vettenranta, 2013), (Prahmana & D’Ambrosio, 2020), and (Rye & Støkken, 2012), a myriad of local contexts in learning activities contribute to enacting the quality of students' learning, providing opportunities to link their education with the problem encountered as a constituent of communities. This viewpoint is also corroborated by some studies from different countries, which argue that local context leads to an improved conception of incorporating STEAM education in mathematics classrooms (El Bedewy et al., 2021)(Kim & Chae, 2016)(Sumarni et al., 2022). For instance, the relevance of local context to students' lives can contribute to students' learning (Holmes et al., 2021)(Morris et al., 2021)(Sevian et al., 2018). In line with the previous notion, authentic tasks were suggested by two studies as opportunities to introduce real-world problems from various fields in genuine situations (Quigley et al., 2019)(Quigley et al., 2017)(Spyropoulou et al., 2020). Hence, the first and second principles proposed are local context and authentic tasks.

The integration of hybrid learning and STEAM education within this study aims to achieve the pivotal goal of introducing adaptable learning in mathematics classrooms. This notion leads to a range of diverse resources, which go beyond an adept understanding of the four core disciplines in STEAM education. Thus, the third principle is STEAM discipline integration. The integration of disciplines refers to the strategies that teachers employ in STEAM subjects, along with the extent to which these disciplines are seamlessly intertwined ((Quigley et al., 2019), (Quigley et al., 2020), (Quigley et al., 2017); (Spyropoulou et al., 2020)).

In the pursuit of enhancing hybrid learning experiences for STEAM lessons emphasizing the engineering design process in mathematics classrooms, it is crucial to make sure teachers develop the instructional harmony of lesson plan components. According to (Moore, 2015), in developing lesson plans, it is essential for teachers to ensure the learning components, involving topic, goals, and objectives; content outline; learning activities, resources, and materials; and evaluation, in particular, to establish a well-structured unit plan (Moore, 2015)(Rusznyak & Walton, 2011). This notion was captured in the fourth principle: instructional harmony.

Through offering activities enriched with digital technologies, teachers can encourage active student participation in the learning process. Additionally, it is crucial to take into account the consideration of accessible and beneficial resources for students’ use (Bush & Cook, 2019). Hence, this was expressed in the fifth design principle, which is digital technologies. In this situation, the hybrid learning methods incorporate remote and face-to-face participation in synchronous learning. Therefore, it is essential to employ the hybrid learning emerging models developed by (Staker, 2011) that adapt learning methodologies encompassing the utilization of digitalization in both offline and online activities, as well as acknowledging the geographical location of students for both in-person and remote involvement. Hence, the corresponding sixth principle is learning strategies of hybrid learning emerging models.

Within the context of learning theory, specifically the theory of didactical situations and didactical contracts, the significance lies in manifestations of scaffolding for students' learning. Hence, it becomes crucial to incorporate the utilization of preceding and following learning activities with the primary goal of enhancing students' preparedness. Preliminary activities serve the aim of introducing problems that will be explored during the main learning activities, while post-lesson activities encourage learning reflection and assessment. These activities can be carried out online through digital technology assistance or in-person through interactive discussion with students as additional activities which are not compulsory. Furthermore, teachers should also formulate guiding questions presented in the learning process, designed to guide students in navigating ideal solutions for the STEAM challenges they encounter. In doing so, guiding questions can be beneficial to encourage students' involvement in actively participating in problem-solving activities in the lessons (Hynes, 2012);(King, 1992); (Council, 2012);(Vasquez et al., 2013). This notion was captured in the seventh and eighth design principles: additional activities with pre-and post-lesson activities and guiding questions.

Moreover, the engineering design process (EDP) can be used as a catalyst for more interconnected STEM education (English, 2016). This idea is also supported by (Brakoniecki et al., 2016), who stated that EDP provides opportunities for students to explore authentic problem-solving activities where they can apply their understanding of mathematics content. Moreover, EDP is beneficial in STEAM education as it can involve students in complex and rich cultural practices (Simarro & Couso, 2021). EDP allows students to conduct prototyping in problem-solving activities ((English & King, 2015);(Jolly, 2017);(Li et al., 2019)), which is also applicable in STEAM activities as part of designing solutions to real-world problems presented in STEAM lessons. Significantly, it can be found that the arts are closely related to engineering, focusing on problem-solving and finding visual solutions through design process activities (Bequette & Bequette, 2012). This notion implies that with EDP teachers can embed the development of students' design thinking not only in technology and engineering practices but also in mathematics and science practices. For that reason, one point that we would like to underline through the design principles is that every student can design and conduct research through EDP not only in science, engineering, and technology but also in mathematics learning. Through the lesson, students are encouraged to employ open-ended strategies for devising solutions, with the intention of nurturing evaluative and innovative cognition through problemsolving activities within the context of the engineering design process (Han & Shim, 2019)(Jolly, 2017)(Council, 2012)(Parker et al., 2017). Therefore, the ninth design principle expresses the main characteristics of EDP, including designing, evaluating, and redesigning.

To offer valuable learning opportunities to students, teachers must emphasize promoting teamwork and facilitate discussion among colleagues. Teachers' collaboration plays a significant role in the implementation of interdisciplinary approaches, including STEAM education (Bush & Cook, 2019)(Liao, 2019). In relation to STEAM learning, the primary role of a teacher is guiding students in collaborative discussions with a primary focus on design activities under teachers' supervision (Quigley et al., 2017)(Singh, 2021)(Spyropoulou et al., 2020). This situation holds a significant factor in discovery. Nevertheless, during the phase of exploration, it remains essential to ensure students also have opportunities to work independently. Therefore, we captured the idea of collaborative work as the tenth design principle.

Next Generation Science Standard, Common Core State Standards for Mathematics, and English Language Arts together urge a variety of assessment tools that focus on formative, diagnostic, and summative assessments (Furtak, 2017)(Vasquez et al., 2013). In this study context, assessment tools that can be utilized in STEAM assessments are checklists, rubrics, classroom tests, maps, self-assessments, peer assessments, graphics organizers, concept maps, portfolios, and others that can be used for formative, diagnostic, and summative purposes (Vasquez et al., 2013). Moreover, (Vasquez et al., 2013) suggested the utilization of self-assessment in STEAM transdisciplinary learning. Hence, the importance of ensuring evaluation which is aligned with learning objectives, including incorporating self-assessment as well as knowledge and skills evaluation within STEAM disciplines, was formulated as the eleventh principle of STEAM assessments.

As the basis of the constructivist learning notion is to encourage students' active involvement in learning, teachers play a crucial part in motivating students to be active as autonomous learners in building their knowledge (Cobb, 1988). For instance, models and questions can be posed as scaffolding for students in problem-solving activities as temporary support (Pol et al., 2010). As a result, scaffolding plays a pivotal role in classrooms, particularly in ensuring students' active participation in STEAM teaching and learning methods. This notion was captured by the twelfth design principle: predictions of students' responses to provide appropriate scaffolding for students during the learning process. Teachers should take into account the act of predicting how students might respond in order to provide relevant feedback. Furthermore, teachers should prepare predictive students' reactions encompassing questions, problems, or other predicted students' answers along with their anticipatory responses (Simon, 1995)(Webb, 2006). This approach equips teachers to empower students through appropriate feedback while students are actively involved in learning (Perry & Lewis, 2009).

Finally, the last principle relates to the enduring long-term understanding of STEAM disciplines. Teachers aspire to establish an enduring comprehension of STEAM fields, fostering a continuous assimilation of the foundational STEAM concepts within students' cognition. This notion involves nurturing a sustained awareness of science, technology, engineering, the arts, and mathematics, ensuring that students are able to employ these skills beyond classrooms. We refer to the suggestions provided by (Vasquez et al., 2013)(Jackson et al., 2021), who encourage the importance of introducing a long-term understanding of STEAM disciplines in classroom implementations.

This study was under the umbrella of a design research project aimed at developing pedagogical guidelines for teachers. Within the design research project, the main investigation was surrounding how the guidelines can support teachers in developing STEAM lessons. (McKenney & Reeves, 2019) demonstrated that the use of educational design research is particularly valuable to contribute to knowledge when there is a lack of existing knowledge, which often occurs in curriculum innovation for quality improvement. According to (Akker et al., 2006), the nature of educational design research is an interventionist, iterative, and practical orientation. Through the iterative cycles, we expected to capture comprehensive and meaningful phenomena in the practical context of design implementation rather than focus on individual variables.

In order to improve the quality of the proposed design principles, our investigation has been continued by incorporating a panel of teacher experts. The activities are in line with (Nieveen, 1999), who suggested the importance of other teachers' and experts' consideration of the materials, especially in terms of perceived usability. Therefore, we employed teacher experts to gather their points of view and recommendations on the design principles. (Jünger et al., 2017) revealed that the significance of employing experts is to ensure reliable recommendations.

In this study, we conducted two screening stages to choose teacher experts as participants. (Palmer et al., 2005) demonstrated how to identify teacher expertise to assist researchers in decisionmaking for choosing experts. The first stage was based on teachers' experience and their educational background. In the first stage, we tried to contact teachers by sending information about our research and asking for their background information, including their consent. In the first screening, teachers should have a minimum of three to five years of experience in teaching mathematics, science, and technology content areas, with a particular population of students.

Moreover, we reflected teachers’ educational backgrounds with aligned certification and degree programs that match their taught subjects. Afterwards, we invited them to indicate their performance recognition by their headmaster or head teachers’ communities and fellow teachers. The performance recognition should be confirmed with a letter of recommendation as evidence, along with documentation on their impact on students’ learning. For instance, a teacher expert should submit a recommendation from his headmaster and his colleagues who were well-acquieted with their teaching knowledge and skills along with the proof of students’ performance. Finally, a total of 80 teachers participated in this study, falling to three groups of twenty-three, twenty-seven, and thirty teachers each. The variability in participation was due to the limited availability of the panel of experts. We summarized the demographic profile of the teacher experts in Table 1.

Table 1 indicates that female participants were dominant, and most teachers had more than ten years of experience. The majority of teachers were experienced teachers aged around 36–45 years. Nevertheless, based on educational background, only 33 teachers possessed a master’s degree, while 47 teachers completed their undergraduate degree education.

We conducted three successive main activities in three rounds over the course of the data collection process following the Delphi method. We considered conducting at least two rounds and reflecting on the rounds to ensure the appropriateness of the activities as well as how to finish the rounds. As suggested by (Hasson et al., 2000), knowing to stop is essential while conducting a Delphi method, which is usually conducted in either two or three rounds. We present our activities in Figure 1.

We held synchronous (two meetings) and asynchronous (a week of individual work) activities in every round. The synchronous activities were conducted utilizing an online conference system, separated by a week of asynchronous activities. In the first synchronous meeting, we discussed the entire process of evaluation, introduced the study's primary objective, and made sure that the teacher experts understood the evaluation process.

We utilized an online questionnaire survey for evaluation, consisting of a Likert scale evaluation for the design principles and an open-ended question for eliciting the teacher experts' recommendations. In the asynchronous activities, we administered a survey consisting of an evaluation and an open-ended question to obtain the teacher experts' recommendations on the design principles. The asynchronous activities focused on individual works on the evaluation of the proposed design principles, which were conducted between two synchronous activities. We asked the teacher experts to provide recommendations on the 13 principles and conducted a focus group discussion in the second synchronous meeting.

The panel experts worked on the online evaluation form to assess the degree of adequacy of the design principles using a 5-point Likert scale (1 = unacceptable, 2 = poor, 3 = average, 4 = good, 5 = excellent). The scoring criteria for the design heuristics are presented in Table 2. As for the open-ended question, we asked the teacher experts to provide their recommendations.

The data analysis focused on qualitative data analysis to investigate the teacher experts’ agreement on the quality of the design principles. The first analysis was conducted on the consensus levels between rounds. We adopted the criteria of the consensus definition by (Mengual-Andrés et al., 2016), as indicated in Table 3. The second analysis was conducted descriptively and qualitatively on the teacher experts’ recommendations on the online form.

In the subsequent section, we organize findings based on the data analysis with presentation of the teacher experts' evaluation of the design principles. The first result focuses on the teacher experts' consensus and agreement in assessing the quality of the design principles.

The subsequent finding concerns the teacher experts' recommendations. We discuss and reflect on relevant literature to build upon contributions on the study implications and future explorations by proposing main dimensions.

According to the results presented in Table 4, the evaluation of the panel of teacher experts has satisfactory levels of proximity and stability. The results showed the median (Me ≥ 4) and relative interquartile range (IQR ≤ 1.5) from three rounds of the Delphi study (Mengual-Andrés et al., 2016). The finding revealed that in the three rounds of evaluation of teacher experts’, the quality of the proposed design principles reached a satisfactory level to aid teachers in the development of STEAM lesson plans.

Furthermore, we analyzed coefficient of variance (CV) differences between rounds to ensure the stability of the teacher experts’ evaluation. Based on the data analysis, as presented in Figure 2, the CV differences between rounds for all principles were below 0.1. Therefore, it can be concluded from the small values of CV that there was stability in the teacher experts’ responses in each round, and consensus was reached, as suggested by (Kalaian & Kasim, 2012).

The teacher experts' positive feedback on the proposed, designed principles revealed that the use of pedagogical support has a significant role in supporting teachers in conducting educational innovations. The importance of pedagogical guidelines in educational innovation corroborates two previous studies conducted by (Janssen et al., 2009) and (Schallert et al., 2022). These studies revealed the positive impact of pedagogical support on teachers' lessonplanning activities. The recommendations from the panel of experts shed light on the importance of pedagogical guidelines for assisting teachers in developing STEAM lessons.

Moreover, the descriptive analysis of the teacher experts’ recommendations revealed several themes, including the trends of suggestions provided by the panel of experts. These themes were employed as opportunities to contribute to future research, which can be beneficial in supporting future explorations to empower the implementation of STEAM education in Indonesia. Figure 3 presents the trends and descriptions of each theme based on teacher experts’ suggestions.

Based on the teacher experts’ feedback, as summarized in Figure 3, we identified significant recommendations for the future explorations and implementation of STEAM education in Indonesia. The panel experts articulated their recommendations under six prominent themes, namely technology usage, assessment, technical matters, contextual problems, integrated STEAM learning, and classroom implementation. Additionally, with respect to the main objective of this study, which was to investigate the quality of the design principles through teacher experts’ recommendations, we discussed our findings and their connections to recent literature to ensure the implications of our study for the field. The following section entails a comprehensive explanation of the findings, followed by a discussion.

As illustrated in Figure 3, according to the teacher experts, future explorations related to classroom implementation hold a significant aspect, enhancing a deeper understanding of the subject matter and making learning more practical in the overall educational experience. By investigating students’ activities during classroom implementation, teachers and researchers might endure the long-term learning impact of their teaching methods. In the Indonesian context, we have many opportunities to work on technology-enhanced learning activities. The result is in line with our previous exploration, which revealed the limitation of technology-enhanced learning activities in Indonesian STEM/STEAM education implementation (Laksmiwati et al., 2023). The findings shed light on the opportunity to also support government endeavour in enhancing the use of digital technologies in Indonesia (Zamjani et al., 2020).

The subsequent finding, based on the teacher experts’ suggestions, is that contextual problems related to local context and authentic tasks are still challenging. Therefore, this finding revealed the need for further explorations of this topic, which could provide a positive contribution to STEAM education. The notion for further explorations corroborates two studies by (Holmes et al., 2021) and (Morris et al., 2021), which revealed the importance of local culture or context closer to students’ lives in the STEAM implementation. Moreover, by utilizing local cultural context in STEAM education, students may have opportunities to explore something relevant to themselves. For instance, Indonesian culture could provide a prospective context for students. The idea is in line with (Ladson-Billings, 1995), who suggested the importance of integrating students’ cultural background and identity into teaching and learning activities through culturally relevant pedagogy. Consequently, future explorations, which opt to provide examples of utilizing local cultural context and problems that are relevant to students’ cultural background, need to be considered.

Subsequently, the teacher experts revealed that the holistic understanding of STEAM education is still a common problem faced by teachers, especially from the perspective of integration. The panel of experts highlighted the essential role of examples of STEAM lessons to illustrate classroom implementation. Therefore, prospective explorations of multidisciplinary, interdisciplinary, and transdisciplinary STEAM approaches ((English, 2016); (Vasquez et al., 2013); (Ladson-Billings, 1995)) can be seen as necessary for future studies.

In summary, the discussion, as mentioned earlier, unfolds a series of opportunities for future studies that can be key domains that need to be emphasized in the implementation of STEAM education. The results of this study revealed the need for pedagogical guidelines, especially in the areas of technology usage, assessment, contextual problems, integrated STEAM learning, and classroom implementation. Moreover, pedagogical guidelines to assist teachers in working through educational innovations, including STEAM education, also need to be taken into consideration in future studies. Thus, the development of pedagogical guidelines might be an initiative to support the implementation of STEAM education in educational settings.

The main objective of this study was to explore and investigate the main dimensions of the proposed design principles in supporting teachers to implement STEAM education in educational settings. We examined Indonesian teacher experts' perspectives and gathered their recommendations, ultimately raising the essential need for the development of pedagogical guidelines for assisting teachers in developing STEAM lessons that are relevant for their students. Therefore, the study emphasizes the significance of the teacher experts' recommendations in guiding future research endeavors, practitioner initiatives, and curriculum development.

Furthermore, the study presented in this manuscript had a twofold objective: to develop design principles for supporting teachers' lesson planning activities. First, the development of design principles might have a practical contribution to empowering teachers' lesson planning skills, especially in promoting teachers' preparation in developing lesson plans within STEAM learning environments. Second, the teacher experts' recommendations encourage researchers and practitioners, as well as curriculum developers, to consider several main dimensions that could contribute to the directions of future studies. The conclusion of this study shows that there is a necessity in future research to utilize the proposed main dimensions in various contexts and consider the pedagogical guidelines in assisting teachers in implementing STEAM. For instance, Indonesia, as the biggest archipelago, offers a unique opportunity for further explorations. In addition, Indonesia's diversity provides various potentials for future research and development, especially in terms of cultural relevance in the local context. Therefore, by embracing future directions, we are intrigued not only by conducting interesting, innovative, and transformative learning but also by broadening the landscape of STEAM education. As our future endeavor, we aim to broaden the landscape of STEAM education, especially in Indonesian educational settings, to encompass comprehensive and diverse perspectives by exploring various classroom experiences and collaborating with teachers. Thus, this study can provide a deeper potential impact on the field of STEAM education, which can lead to positive changes in educational practices.

Author Contribution:

Author 1: Research design and manuscript writing; Authors 2 and 3: Review and supervision.

Funding Statement:

This study was funded by the Austrian Federal Ministry of Education, Science and Research (BMBWF) with reference number MPC-2022-06063.

Conflict of Interest:

The authors declare no conflict of interest.

Additional Information:

All data are available for this paper.