Evaluation of Computer Tools for Idea Generation and Team Formation in Project-Based Learning

Hemos publicado en la revista Computers and Education (IF=2.059) el resultado del proyecto Ideación2.0.

El articulo completo esta en: http://dx.doi.org/10.1016/j.compedu.2010.10.012

Cito las parrafos y figuras más importantes:


The main objective of this research was to validate the effectiveness of Wikideas and Creativity Connector tools to stimulate the generation of ideas and originality by university students organized into groups according to their indexes of creativity and affinity. Another goal of the study was to evaluate the classroom climate created by these tools and the method “Think Actively in a Social Context” (TASC) proposed by Wallace and Adams (1993) and framed within project-based learning (PBL). The research was conducted with a sample of 34 students in the third year of a Computer Engineering degree, which, during a period of 15 weeks, required them to design and implement an innovative distributed application project. The procedure consisted of the implementation of the eight phases of the TASC method integrated to the Wikideas and Creativity Connector tools. The information provided by the tools, interviews and questionnaires administered to students was used to analyze our hypothesis. The results show that the tools helped the students to generate, evaluate and select the most relevant ideas and to form teams for project execution. They also revealed that teams with high indexes of creativity and affinity (type α) achieved the best grades in academic performance and project originality. Furthermore, research data show that Wikideas and Creativity Connector along with the TASC approach created a positive classroom climate for students. Based on this work, several suggestions can be extracted on the use of the tools and the TASC method for project-based learning.

2.1. Wikideas

Wikideas facilitates the generation of ideas in nominal groups and small groups, as well as the evaluation of ideas in brainstorming networks generated by the Creativity Connector tool. It is a Web-based interactive tool based in the WakkaWiki software, which provides functionality beyond other electronic brainstorming systems (Prante, Magerkurth, & Streitz, 2002). It is very easy to use because every idea is transformed into a Wiki page and has four main functions: generating, communicating, analyzing, and assessing information.

1. Generating ideas. This task is performed in a digital private space, in which each user can develop, express, and store his/her new ideas as private Wiki pages containing a detailed description of the idea and can be revised as many times as necessary to improve it.

2. Communicating ideas. Information saved in the above-mentioned private space can be published anonymously so that other users can have access to them. This task is carried out in the following way. After clicking on the information a user wishes to share, the user drags and drops it onto a panel of public ideas as indicated by an icon that symbolizes shared idea. From then on, the information can be viewed by other application users.

3. Analyzing ideas. Published information can be commented on and examined by other users, who can then formulate questions anonymously. Then in a low-risk environment, the author can answer these questions in the corresponding Wiki page, either redefining the idea or adding a pertinent observation. The ideas that have been commented on by other users are marked on the “idea generator” panel so that their creator can respond to the comments.

4. Assessing ideas. Using a Likert scale that range from 1 (very small interest) to 5 (very high interest). Each user can pick up to a pre-specified number of ideas that he or she considers the most interesting.

Before team groups are created, ideas, comments and ratings are shown to other participants without any reference to its author; when final project teams are formed, names of authors appear alongside ideas, comments and ratings. All information relevant to each function can be visualized in its corresponding panel. Figure 1 shows the interface of the initial Web page of this tool, which can be used to access the panels of each functions mentioned above.

2.2. Creativity Connector

Creativity Connector is a social network tool that connects users who wish to participate in common projects. It is implemented using graph algorithms and a collaborative filtering system. This tool is integrated with Wikideas and uses the information generated by its users to relate and bring together participants. The first main function of Creativity Connector is to create a brainstorming network whose nodes are users of Wikideas, which are connected to N other participants. This network is used to show each participant’s ideas only to those participants who have a link in that network. The second main function is to form creative affinity teams. Creativity Connector performs both functions automatically and presents results networks and teams graphically as shown in Figure 2, in which 9 groups can be observed according to different indexes from affinity (major thickness of the lines represent major index of affinity). The class instructor can inspect the resulting teams and manually make some adjustments. The implementation of these functions is defined in the following way.

Brainstorming network creation. After idea generation sessions, the tool calculates the creativity indices of participants using the number of ideas proposed by each participant and the amount of development each participant has done for each idea by measuring the length of the description of each idea. Then the tool assigns to each participant a numerical value proportional to both quantities. The tool creates a brainstorming network connecting each participant with N users that have a similar value of creativity. In practice, each user is connected to N/2 participants with a higher creativity index and N/2 participants with a lower creativity index. This network is used to show each participant’ ideas only to those participants who have a link in that user’s network.

Formation of creative teams. To form creative teams, Creativity Connector makes use of the rating values assigned by each user to different ideas in Wikideas. With these values, the tool calculates the affinity between participants so that two participants who have similarly rated the same ideas will have a high affinity (Resnick & Varian, 1997). These pair-wise affinities are used to establish related teams in terms of the sum of pair-wise affinities between the members of the team. Team affinities are used to form teams for project development. These teams are selected from all possible combinations of teams. First, the team with the highest affinity among its participants is stipulated and then users in that team are removed from the pool of users used to form teams. This process continues until all participants are assigned to a team.

Figure 2. The graphical user interface of Creativity Connector to visualize affinities of brainstorming networks and teams generated automatically

3.         Purpose of the study

On the basis of the theoretical and empirical framework discussed previously, this work’s main objective was to validate the Wikideas and Creativity Connector tools’ ability to sustain the production of ideas, originality and formation of teams for project-based learning. The following three hypotheses were formulated.

First, Wikideas and Creativity Connector tools will support generation of creative ideas and originality by university students in a project-based learning course. With this goal in mind, several new functions beyond standard EBS systems (Prante et al., 2005) have being designed: 1) maintaining anonymity of ideas, comments and ratings, 2) calculating creativity and affinity indexes of participants and 3) the configuration of a network of brainstorming peers to limit the number of ideas that each subject can see and evaluate.

Second, teams formed by subjects with high indexes of creativity and high indexes of affinity between them (type α) would obtain better results in originality and academic achievement than those teams with low indexes of creativity and high indexes of affinity (type β), high indexes of creativity and low indexes of affinity (type γ) and low indexes of creativity and low indexes of affinity (type δ). This hypothesis was proposed because, according to the literature reviewed, the complex tasks that are involved in a challenge to the members of a team require high levels of creativity and a strong intrinsic motivation (Bantel, 1994; Bowers, Pharmer, & Salas, 2000; Uzzi & Spiro, 2005). We guessed that the planning and implementation of an innovative distributed application project is a difficult task for students in their third year. Teams will be created by Creativity Connector using participants’ creativity and affinity indexes obtained in idea generation and evaluation sessions.

Third, Wikideas and Creativity Connector tools adequately integrated into the project-based method called “Think Actively in a Social Context (TASC)” will create a positive classroom climate that will help them to achieve the aims of the academic course and to create an original project. This environment will influence favorably the intrinsic motivation of the students, their teamwork and their learning.

4.         Method

4.1.      Participants

The subjects of this study were 34 students from the Public University of Navarre Software Engineering Degree Program enrolled in the third-year course entitled “Advanced Operating Systems” during a semester-long (15 weeks) period for the academic year 2009-2010. The student’s mean age was 20.3 years and the standard deviation was 1.1. In addition, 30% of participants were female and 70% were male. Eleven project groups were formed, each of which had three members except one with four.

4.2.      Procedure

The procedure consisted of the implementation of the 8 phases of the method TASC proposed by Wallace and Adams (1993). Figure 3 shows the sequence of the phases, the tools used, the form of work, and the duration of every phase. The following are descriptions of the tasks that the teacher and the students performed in the different phases.

1. Gathering and organizing. On the first day of class, the teacher explained to the students the following points: (a) the purpose of the course entitled “Advanced Operating Systems”, (b) the importance of creative development for the engineering profession, (c) the use of two tools, Wikideas and Creativity Connector, which would help in achieving the course objectives, (d) the method that was used in the course called “Thinking Actively in a Social Context, TASC” that also would help them in creativity, (e) how learning in the course would be evaluated, and (f) that to achieve these goals, they should work responsibly, both individually and as a group. Then the teacher conducted some brainstorming exercises about several topics, for example, “how can you use a Web search engine?” to familiarize to students with this technique. They had to undertake these reflexive and creative tasks using materials provided by the teacher and Internet information sources such as Wikipedia and Google.

2. Setting goals. In the spaces provided by Wikideas, the students, with the help of the teacher, defined the course goal, which was “the design and performance of a creative software product that was a distributed application composed of at least four different subsystems: a Web robot, a processing server, a storage module, and a Web server”. In addition, they also fixed the evaluation criteria that they would use to assess the obtained results, namely, the student’s active participation in both individual and group activities, implementation of a new project that could have some usefulness outside of the class, and defense of that project in front of the class.

3. Generating ideas. The students, individually, proposed ideas about possible projects using the “generate ideas” function of Wikideas. After this step, the Creativity Connector tool was used to create a brainstorming network so that students could see ideas from eight different students.

4. Evaluation of ideas. In this phase, students communicated their ideas to their classmates anonymously and analyzed their classmate’s ideas using a score of 1 to 5. Then the Creativity Connector tool used those ratings to form three-person project groups according to affinity. After this phase, the teacher conducted the first interviews of the study.

5. Developing the Project. Organized in groups according to their creativity indexes and interests, the students monitored the execution of their project, always taking into account the fact that it should display clear signs of originality. During this stage, each group met with the teacher weekly to report on their work, to clarify doubts and to specify the tasks to complete for the next meeting. To carry out this task, to communicate with the classmates, to exchange files, and so on, students also used other tools, such as Wikis, e-mail, and Blogs.

6. Assessment. The students reflected upon their own work and their participation in a small group. Then they sent their conclusions to the members of their group. Later, working together, they drafted a comprehensive assessment of their workgroup.

7. Presenting the projects. In various class meetings, each group defended its project in front of the class, underlining its achievements with regard to creativity. They also compared their results with those of other groups and commented on the help provided by the computer tools and the limitations observed in the execution of the project. This phase allowed other students to see what their course mates had done and to provide feedback to them where necessary. The groups carried out this activity using additional tools, such as Video Editing and Presentation software, which allowed them to present their work in a professional manner.

8. Learning from experience. The students reflected on how to transfer their learning to society and businesses. They commented in detail on the aspects of the project and the tools that should be improved in future experiences and on the significant changes achieved in their creative competencies. In the end, students responded individually to the questionnaire.

During all of these procedures, the teacher’s task was to provide guidance and to set up some criteria to help the students complete their projects. They would also come to the teacher for advice on technical as well as design problems. This provided the teacher with the opportunity to monitor their progress and to make sure no team was left behind. Some class lectures were converted to consultation periods for both the students and the teacher to gather feedback from one another. The role of the teacher was that of a facilitator.

Figure 3: The phases of the TASC method, tools used, form of work, and duration of the phases

6.         Discussion

In this study, we intended to validate the effectiveness of Wikideas and Creativity Connector tools to support idea generation and originality in university students organized into different project teams, taking into account their levels of creativity and affinity. The study also aimed at evaluating whether the classroom climate created by the two tools and the project-based learning (PBL) method called “Thinking Actively in a Social Context” was appropriate to carry out an innovative project and, consequently, to achieve the goals of the course.

6.1.      Idea Generation

The results suggest that the tools promoted both the generation of ideas in students and originality in the development of a software project, which consisted of a distributed application composed of a Web robot, a processing server, a storage module and a Web server. This statement is based on information provided by the tools, the questionnaire and the project accomplished.

First. The tools helped students in the production of ideas, in the visualization of a large number of ideas of the other students (M = 65), in the assessment of some of the ideas (M = 11), and in the selection of the best idea for subsequent implementation in a project. The students expressed that in relation to this discrepancy between the number of ideas they had accessed and the number of ideas selected by them, many of the ideas of their colleagues were either very similar or seemed too difficult to implement. These tasks were performed by students in private and public digital spaces in conditions of anonymity, allowing them to choose the ideas of their colleagues that seemed to be the best regardless of their friendship. In this context, a student said, “At first the task was difficult, but gradually, with the help of the tools, the ideas flow in my mind and immediately wrote them in the generate ideas panel with great ease.”

Second. The results of the questionnaire revealed that students had highly positive perceptions of the support they received from the tools for generating ideas, communicating their own ideas to others, evaluating the ideas of other students and developing all project stages.

Third. The assessment carried out by researchers on the originality of the project also supports the assumption that the tools helped students to execute their tasks with a degree of originality, a level of complexity and some interesting applicability in society. The contribution of the above three sources allows us to assert that the first hypothesis of the study was confirmed.

These results point in the same direction as other studies that have researched creativity and creativity support by computer tools (Florida, 2002, 2005, Jang, 2009, Johnson et al., 2002, Li, 2002, Shneiderman, 2007), although they surpassed some previous findings. For example, the traditional brainstorming phase ends with the generation of new ideas (Chidambaram & Tung, 2005; Dornburg et al., 2009; Michinov & Primois, 2005). In contrast, our study participants also had to design and implement a project with the best ideas selected; this resulted in students discussing whether ideas could be implement when they could see each other’s ideas, instead of generating more ideas as is reported in studies which consider brainstorming as an end in itself. Being able to determine the number of ideas a user has to visualize and evaluate without interfering with his creative process, is another advantage offered by our tools. However, the best mechanism still remains in debate. We conducted a pilot study testing other configurations of tools so that participants could see a different number of ideas. In the current version, they could see the ideas of eight companions; however, both studies only evaluated a small number of them. We hope to provide to the scientific community more definitive data about this in future publications. Moreover, the tools we developed automatically calculate indices of creativity and affinity; to our knowledge, this has not been provided by other tools, which are used to evaluate the productivity of a subject and establish project teams. As a consequence, our tools have resulted, as stated by Shneiderman (2002), in students working collaboratively and expressing individual and social creativity in new ways.

6.1.      Team Formation

Wikideas and Creativity Connector formed four types of project teams, which consisted of the following groups: (1) subjects with high levels of creativity and affinity (type α), (2) subjects with low levels of creativity and high levels of affinity (type β), (3) subjects with high creativity and low levels of affinity (type γ) and (4) subjects with low levels of creativity and affinity (type δ). Of these categories, we postulated that type α would be better in originality and academic achievement than other categories. The data of Table 1 and the results of the survey and questionnaire, support our second hypothesis. That is, students with high levels of creativity and affinity were able to propose a more innovative project, implemented the components recommended by the instructor, sought the practicality of the idea and presented the project before their peers well.

Students themselves recognized these in their assessments, commenting that “the fact that all members of the team had a lot of ideas and the motivation to achieve the same goal helped us to work in a coordinated way to implement the project in an original way.” It, thus, appears that the homogeneity of a team about their ideational creativity and an interest in a common goal can be an important factor to ensure the implementation of an original project, as highlighted by Bantel (1994) and Bowers et al. (2000).

Another important result achieved was that type γ project teams formed by high creativity and low affinity students also achieved very good results in originality and academic performance, which suggests that, once accepted, the members of the teams who came up with the project idea were motivated and worked responsibly and with originality in their development. These data show that students can change their interests and thus direct their efforts towards achieving new goals to compete successfully with other groups (Paulus & Brown, 2003). The tools helped these project teams to avoid spending too much time and effort in analyzing which idea they could choose to implement in their project and was considered an important advantage in the study of Yang and Cheng (2010).

Most participants said in interviews that they were satisfied with the structure of the team and only a few were a bit uncertain in working with partners who initially did not share the same interests. However, as mentioned, this feeling changed for the better with the help of the teacher and the receptive attitude of the students. Therefore, these students needed more guidance at the early stages of development of the project, but soon the team began to function properly. Similarly, the results provided by the questionnaire confirmed that the teams formed by the tools were highly accepted by students.

The data of Table 1 also indicate that tools were able to arrange teams in clearly defined categories in term of the average creativity and affinity indexes of teams. However, the resulting number of teams in each category (3 type α, 1 type β, 3 type γ and 5 type δ teams) indicates that it is difficult to form many teams with high levels of affinity and creativity among members, as was expected. Furthermore, a significant proportion of the teams with low levels of affinity and creativity among members were formed. Organizing small teams out of a large group is a complex problem that has no ideal solution if too many restrictions are applied; this is likely the scenario we confront in the case of our study. Our grouping strategy might have to be evaluated in the future, but if we can classify the resulting teams in terms of a small number of variables as we did, it is easy to provide guidance tailored to each team.

6.3.      Classroom Climate

The tools and the “Thinking Actively in a Social Context” method created a positive classroom climate for students. Indeed, the integration of both resources, tools and methods, offered continued support in implementing the project, helped them overcome the challenges in every step of the learning process, prepared them for creative action by using the tools in the steps of the method that best encouraged creativity and encouraged the interaction between the learning social context and the exercise of their cognitive processes. A classroom climate with these characteristics could contribute to all students achieving their academic objectives. As the data in Table 1 indicate, teams finished their projects with fairly high grades. Type δ teams, which were composed of low creative members with low affinity among them, performed well and passed their projects. They commented, “we have learned many things during this academic year, one of which is how to work in a team, which prepares us significantly for professional practice.” These results support our third hypothesis as well as the fact that it is also expressed by participants in the questionnaire.

These results corroborate the theories of Adams (2006), Laffey et al. (1998) and Nagel (1996) that relate PBL with an active, responsible and collaborative participation of student in learning and “learn how to learn” characteristic to the constructivist perspective. Moreover, as some researchers have emphasized (Sanz de Acedo et al., 2009, Wallace & Adams, 1993), the effect of the TASC method in creativity skills has proven to be significant. We have observed, like many researchers (Anderson, 2002; Hunder et al., 2007, Laffey et al., 1998; Yand & Cheng, 2010), that the classroom climate and method of instruction used is vital to encourage students in developing their creativity.

Un comentario en “Evaluation of Computer Tools for Idea Generation and Team Formation in Project-Based Learning


Introduce tus datos o haz clic en un icono para iniciar sesión:

Logo de WordPress.com

Estás comentando usando tu cuenta de WordPress.com. Salir /  Cambiar )

Google photo

Estás comentando usando tu cuenta de Google. Salir /  Cambiar )

Imagen de Twitter

Estás comentando usando tu cuenta de Twitter. Salir /  Cambiar )

Foto de Facebook

Estás comentando usando tu cuenta de Facebook. Salir /  Cambiar )

Conectando a %s