Learning in vivo

Project based learning (PJBL) as per the SA Council on Higher Education

The CHE’s Good Practice Guide (2011, 4) on WIL advocates the “fostering [of] university learning that is less didactic and more situated, participative, and ‘real world’ oriented”. The integration of theory and practice may occur through a varied range of approaches, such as formal or informal work placements, “action-learning, apprenticeships, cooperative education, experiential learning, inquiry learning, inter-professional learning, practicum placements, problem-based learning, project-based learning, scenario learning, service-learning, team-based learning, virtual or simulated WIL learning, work-based learning, work experience, workplace learning” (emphasis added). An underlying commonality—not restricted to placements—is the alignment of academic and workplace practices for the mutual benefit of students and workplaces.

The CHE’s Good Practice Guide (2011, 18) qualifies that “project-based learning (PJBL) involves learning through projects”. These projects could either be ‘real’ or simulated. ‘Real’ projects are undertaken in a workplace and generally involve “elements of research and the supervision by both a university teacher and workplace supervisor or mentor”. Projects “are a means of engaging students in complex, work-related issues, through which they develop and transfer skills and knowledge”. It is important to point out that “projects do not always lend themselves to ‘coverage’ of all outcomes in a curriculum. This means selecting those topics that reflect the most important ideas and concepts in the curriculum and incorporating those topics into projects.”

The application of project-based learning is discussed at the various HEQF levels, namely:

For level 5, the one-year certificate (2011, 23), it is not recommended because “literature recommends that PJBL components be approximately one semester in duration”.

For level 6, the two-year advanced certificate (2011, 24), only short focussed projects.

For level 7, the three-year diploma/Bachelor’s degree (2011, 24), “A focused project, such as a service learning project, would be appropriate at the third year level, provided that there are adequate resources for such a project – in particular a transport budget, time allocation, and formal agreements with community partners and service providers” (emphasis added).

For level 8, the four-year professional Bachelor’s degree (2011, 25), the same condition regarding adequate resources is expressed, however “formal agreements with workplaces, the provision of mentors/coaches” is mentioned. Further stated that “PJBL is recommended in the third and fourth year” because “PJBL can contribute significantly to student learning”, “when the students are more able to learn from complex situations, have near-professional level skills, and are thus able to contribute to problem solving or innovation in a workplace in a meaningful way.”

For level 9, the Master’s degree (2011, 25), the theses are generally of an applied nature, meaning PJBL research “in which an intervention or product is designed, implemented, studied, and evaluated” could be ideal.

For level 10, the Doctoral degree (2011, 25), the same applies as for level 9 but “with particular emphasis on the contribution to knowledge”.

The engagement of students in PJBL teaches such “students complex processes and procedures such as planning, managing and communicating”. However, the CHE’s Good Practice Guide (2011, 36) cautions about the alignment of this pedagogical approach that the accomplishment “requires time for both teachers and students to master the strategies necessary for successful PJBL”. The reason is that “project work involves many different types of projects within the framework of a single subject it can be demanding on staff resources”.

The CHE’s Good Practice Guide (2011, 43) regards project-based learning as ‘high stakes’ assessment, because it often counts proportionately more marks than theoretical modules. Historically students kept “logbooks or build portfolios to show their experience and the skills development”. However, “these paper-based forms of assessment are” considered “inadequate in terms of demonstrating students’ mastery of complex practice”. The CHE’s Good Practice Guide (2011, 43-44) presents an example of video-diaries used rather than paper-based assessment. It further (p. 48) points out that “clarity on the assessment process in project work is probably more important than with traditional programmes because of the demands it makes on academic staff.”

The CHE’s Good Practice Guide (2011, 54-55) point out that project-based learning “has considerable potential for advanced student learning and for solving ‘real life’ problems in workplaces.” However, it also cautions that “there are extensive resources [needed] for initiating, building and maintaining community-based partnerships in the CHE guide on Service-learning in the curriculum (2006)” for enabling students to “assist a community in need, using knowledge and skills gained in their course of study”. Similarly, is the “support of university teachers” necessary to “address a real workplace problems”. The CHE’s Good Practice Guide (2011, 54-55) differentiates the roles of the internal/academic partners and the external/professional partners with regard to project-based learning. The lecturer “takes on the role of a professional consultant”; “usually plays a significant role in addressing the workplace problem or need” and “should model professional conduct”. The “students will take on a professional identity, which is an important aspect of their development as they learn to deal with clients in a professional manner”. The “workplace partners play a major role” in identifying “a particular need or area of work that requires development and, in collaboration with the academic partners, define and shape the work so that there is a match between workplace requirements and the academic programme”.

The academia/lecturers’ role	The workplace professionals role
Assume a professional identity,   and explain issues in professionalism Act as consultants Supervise the students engaged   in the project Ensure that the students meet   regularly with their ‘clients’ Monitor the progress of the   project Assess the final product or   process, in collaboration with the professional partner	Invite students to solve an   authentic workplace problem Provide students with sufficient   orientation to the workplace and the particular problem Arrange adequate time for   students to consult with workplace supervisors or mentors Assist with the assessment of   the final product or process, in collaboration with the academic partner

The CHE’s Good Practice Guide (2011, 61) points out that the management of project-based learning requires “considerable planning, as well as relationship building with workplace partners.” It recommends formalising the partnerships with workplaces by entering into a memorandum of Understanding which clarifies the respective roles and responsibilities of academia, the workplace partners and of students, but also address health and safety issues.

The workplace may be involved with the selection of students, their orientation to the workplace in general as well as to the specific project, etc. It is important that full records are kept, including workplace and contact person details, student lists (which projects they are involved with, contact details, etc.). Data capturing and management of the database of students and workplace would probably be done by administrative staff, but should be overseen by the academic manager or university teacher who has the main responsibility for the project. PJBL involves liaising with workplaces; this might include attending meetings between students and their ‘clients’ (although this could be delegated to tutors or mentors). Students’ progress on their projects should be monitored, regular feedback given, and their work should be formally assessed, usually in collaboration with workplace partners.

The CHE’s Good Practice Guide (2011, 61) admits that “creating and managing authentic opportunities for projects in a real work context is a time consuming task and opportunities for student work placements might be limited in particular fields.” It therefore state that project-based learning is often simulated “at the university in order to enable students to experience some aspects of the workplace within an educational framework.”

Republic of South Africa. Council on Higher Education (CHE). 2011. Work-Integrated Learning: Good Practice Guide. Higher Education Monitor 12, August 2011. Pretora: IeCommunications. http://cctprojects.co.za/wbeproject/documents/WIL/Higher_Education_Monitor_12.pdf or http://digitalknowledge.cput.ac.za/handle/11189/5205

Inductive and problem-based pedagogies are used in workplace learning

Inductive pedagogy, meaning learning starts with real-life (in vivo) experiences or examples, upon which theory is built or experiences from which theory is extrapolated. Problem-based pedagogy reinforces learning in that students use both prior experiences and theoretical knowledge to solve problems and challenges. Nuninger and Châtelet (2011, 93) argue that inductive and problem-based pedagogies is generally much better than deductive pedagogy, where theory is first taught and then applied. Errors made are acceptable and correction takes place through reflective practices.

Nuninger and Châtelet (2011, 88) emphasise the importance shared responsibility between university and industry for inductive pedagogy; especially with regard to mentoring/tutoring and the assessment of skills proficiency. Academic knowledge is generally evaluated by lecturers, whereas workplace competencies jointly evaluated by industry and academia.

Nuninger, W. and Châtelet, J.M. 2011. Work-Integrated Learning for Engineers in Coordination with Industries. In P. Keleher; A. Patil, R.E. Harreveld (eds). 2011. Work-integrated learning in engineering, built environment and technology: diversity of practice in practice (pp. 85-109). Hershey, Pa.: Information Science Reference.

 

Importance of precise observation of skills demonstrated

Kogan et al (2011) emphasise the importance of precise observation of skills demonstrated and of giving first-rate feedback as essential aspects for the development of competence.

In clinical training many tools have been developed and published for the direct observation of the skills of residents with patients (Kogan et al, 2011). Despite this, performance ratings are often subjective and subject to rating errors, such as inaccuracy and unreliability. However, there has not been much investigation into causal factors of rater variability. Explanations by raters themselves for low inter-rater agreement include “differences in observer gender, ethnicity, experience or clinical competence” (p. 1049). In a study by Kogan et al (2011: 1051), four themes were identified to explain
the variability of judgements and ratings, namely:           
(i.)        “the use of variable frames of reference during observation and rating;        
(ii.)         the role of inference;       
(iii.)        the use of variable approaches to synthesising judgements into numerical ratings; and       
(iv.)        factors external to resident performance that influence ratings.”

Kogan et al (2011: 1051) found that many direct observers use their own outlook—how they do things—or what they consider a respected practitioner would, as criteria for making judgements and giving ratings.

Based on the four themes, Kogan et al (2011: 1056) created a descriptive model of the direct observation process. Present within observers are an amalgam of characteristics, values, expertise and emotions that could impact their observations. Observation is done through two lenses, namely the frame of reference as yardstick lens; and the other lens represents inference, which shapes the meaning and interpretation of the observations. The observation is furthermore done within a context of practice. The observations and interpretations are synthesised into a rating. Unfortunately reality is not neatly, foreseeable or straightforward. There are often other influences and complexities present that may sway ratings. Feedback to the observer, further contribute to modify current and future ratings. The model can be supported by situated cognition theory, namely “an individual’s thinking, knowing and processing are uniquely tied to and inextricably situated within (and cannot be completely separated from) the specific social situations within which those thoughts and actions occur” (pp. 1056-7).

Kogan, J.R., Conforti, L., Bernabeo, E. Iobst, W. & Holmboe, E. 2011. Opening the black box of clinical skills assessment via observation: a conceptual model. Medical Education, 45, 1048–1060.

Regulations about Skills Development Funds received and spending

On 1 April 2013 new regulations (RSA 2012) about monies received by Sector Education and Training Authorities (SETAs); mandatory and discretionary grants; proportion of administration costs; discouraging accumulation of surpluses; and related came into effect. Simultaneously the previous grant regulations were repealed, namely Government Notice No R713 in Government Gazette No 27801 of 18 July 2005 and Government Notice No R88 in Government Gazette No 29584 of 2 February 2007.

Of particular importance to higher education programmes that contains work-integrated learning in the curriculum is §6 (pp. 11-13), “allocation of discretionary grants by a SETA”:

• §6(12)—“at least 80% of discretionary grant funding must be allocated to PIVOTAL programmes”, which are defined (p. 5) as an acronym meaning “professional, vocational, technical and academic learning programmes that result in qualifications or part qualifications on the National Qualifications Framework …” [also §3(6)]
• §6(11)—addressing scarce and critical skills through programmes designed to address such, which include work-integrated learning must be the key focus of SETAs. Work-integrated learning is defined (p. 6) as an umbrella term for curricular, pedagogic and assessment practices; integrating formal learning and workplace concerns; culminating in a qualification or part thereof; and may include practicums, professional practice, internships, workplace experience, industry-based learning, cooperative education, service learning, real work learning, placements, experiential learning, and clinical placements
• §6(14)—obtaining the funding above is subject to a legal person completing and submitting a PIVOTAL training plan and report as per annexure 2 (private), sections E & F (pp. 22-23) or (public), sections G & H (pp. 30-31). A legal person [§6(10), p. 12] includes public education and training institutions; an employer or enterprise that falls within the jurisdiction of a SETA, but includes those not required to pay the skills levy; and other such as NGOs [refer to §6(7), p. 12].
• §6(15)—however, employers with less than 50 employees will not have to submit the PIVOTAL training plan and report indicated above, “but will be asked to provide information on participation in PIVOTAL programmes and the impact of these”. SETAs will develop suitable templates for small and micro enterprises.
• Note, however, §7 specifies that the “SETA criteria for grants must be approved by the SETA Accounting Authority before funds are allocated”; and §8 that each SETA is required to prepare and distribute a schedule indicating the dates by which applications for grants must be submitted and the timeframes for disbursements [also §3(7)].
• §6(1-2 & 4-5)—each SETA must develop a Sector Skills Plan (SSP) in accordance to the National Skills Development Strategy and strategic goals; and each SETA must annually approve a Discretionary Grants Policy, developed in accordance to the issued DHET guidelines; which will determine and specify its discretionary grants allocation to meet the sector’s needs through different delivery models and project types that will be deployed.
• §6(3)—each SETA’s Annual Performance Plan (APP) clarifies how discretionary grants are allocated to prioritise skills development of the sector through public education and training institutions.
• §6(9)—the Discretionary Grants Policy of each SETA must “indicate the purpose of the various grants, funding frameworks and projects …”; and give the communication procedures, which must embrace the “principles of transparency, opened, access and fairness” [§6(8)].

The remainder of the regulations deal with (§1) definitions; (§2) specifying the limitations on administrative costs of SETAs; (§3) the compliance of SETA finances and financial management; (§4) the allocation of mandatory grants, and (§5) the payment mandatory grants to levy paying employers; (§9) the payment of services rendered by SETAs; (§10) the repeal of previously published regulations; and (§11) finally the short tile and commencement of the regulations.

Republic of South Africa. 2012. Skills Development Act (97/1998). The Sector Education and Training Authorities (SETAs) grant regulations regarding monies received by a SETA and related matters. Government Gazette Vol. 570; Regulation Gazette No 9867; Government Notice No R990; Gazette No 35940; 3 December 2012. The Government Printing Works.

Gazetted Terminology for Professional Nurse and Midwife Education and Training

The terminology for cooperative and work-integrated education within the South African nursing education and training sphere is worth noting. It is gazetted that the Minister of Health, after consultation of the South African Nursing Council (SANC)—a statutory body—in accordance to §58(1)(f) of the Nursing Act (Act No 33 of 1995); made regulations pertaining to the education and training of professional nurses and midwives.

Of specific interest are the definitions of:

• Clinical accompaniment, which “means a structured process by a nurse education institution to facilitate assistance and support to the learner by the nurse educator at the clinical facility to ensure the achievement of the programme outcomes;”
• Clinical facility, which “means a continuum of services to promote health and provide care to individuals and groups, used to teach learners;”
• Clinical learning opportunities, which “means the range of learning experiences available in a healthcare setting or other experiential learning sites for a learner to gain the required clinical skills;”
• Clinical placement, which “means the period spent by a learner in clinical and other experiential learning sites to ensure that the purpose of the programme is achieved;”
• Clinical supervision, which “means the assistance and support extended to the learner by the professional nurse or midwife at a clinical facility with an aim of developing a competent, independent practitioner;”

The gazetted regulations further includes several other definitions of terminology such as assessment / assessment method / integrated assessment / assessor / moderation / moderator; competence / learning outcomes / programme outcomes; comprehensive nursing; fundamental learning; programme / qualification; and recognition of prior learning. The regulations further deal with issues such as requirements for registration in the two said nursing categories; accreditation of education institutions; admission criteria; minimum programme requirements; clinical education and training; the assessment and moderation of learning outcomes; transfers; recognition of prior learning; application of these regulations, as well as transitional arrangements.

Nursing Act: Regulations: Approval of and minimum requirements for education and training of learners leading to registration in categories professional nurse and midwife, (G 36235, RG 9927, GoN 174), 8 March 2013

Eight categories of assessing work-integrated learning

Sonia Ferns and Keri Moore (2012) indicate that assessment of fieldwork generally comprise three broad elements, namely (a) actual workplace performance; (b) a record of the experience (diary or journal); and (c) a reflection on the work-based experience. However, they mapped eight categories of assessment in use for assessing student outcomes from fieldwork placements:

• Case Studies research on cases: applying disciplinary knowledge in a professional context, roles, ethics, and responsibilities, systems, processes and lateral and higher order thinking, debates. Can be inter-professional or discipline specific.
• Reflection/portfolio: Reflection of work place experience and collection of evidence.
• Simulation: Institutional based assessment intended to replicate the workplace environment e.g., role play including Objective Structured Clinical Examinations (OSCE), peer and self-assessment.
• Knowledge test: Testing recall of facts such as online quizzes or multiple choice quizzes (MCQ).
• Performance - real-time in situ: Practical assessment of work place performance - detailed understanding of disciplinary systems, roles, ethics, and responsibilities, systems, processes and lateral thinking, proficiency in basic disciplinary skills, communication and core disciplinary knowledge.
• Oral Presentations: oral test - detailed understanding of disciplinary systems, roles, ethics, and responsibilities, systems, processes and lateral thinking.
• Written task: Essays or reports (other than case studies) higher order thinking.
• Industry-based case study: Investigation of actual work place scenarios.

Their research findings include, among others, some interesting graphs about frequencies of use of the above mentioned assessment types; and of graduate attributes assessed.

Ferns, S. & Moore, K. 2012. Assessing student outcomes in fieldwork placements: An overview of current practice. Asia-Pacific Journal of Cooperative Education, 13(4), 207-224.

E-portfolios and assessment via mobile technologies

The South African Council for Higher Education (CHE) imperatives with regard to programme (qualification) accreditation include, among others:

• That the programme must bring about an understanding within students about the specifics of the occupation for which they are being trained.
• That the students should master the techniques and skills required for a specific profession or occupation.
• And where feasible that work-based learning and placement in a work-based environment should form an integral part of the curriculum.

The CHE imperatives with regard to work-integrated learning further include (1) a mentoring system that enables the student to recognise her/his strengths and weaknesses in his/her work, to develop existing and new abilities, and to gain knowledge of work practices; as well as (2) regular and systematic monitoring. Mobile devices may well enable both these.

However, Coulby, Laxton, Boomer and Davies (2011) caution that without a sound pedagogical rationale for the use of mobile devices; technical and infrastructural realities may very possibly reduce student engagement.

Higher educational health and social care curricula often comprise a blend of classroom instruction (in residential education); simulations and work-based placement/s. The latter provide for learning opportunities and the assessment of competencies and professional attributes that cannot be cannot be replicated in the former two. The authentic nature of work-based placement/s enables holistic preparation; preparing graduates for professional practice, including the culture and practices of the particular community of practice. Primary care community clinics are utilised for placements in teaching. (Coulby, et al citing Billet, 2001; Lave & Wenger, 1991; Vygotsky, 1978).

Although work-based placements are regarded essential; Coulby, et al realise that these learning and teaching environments are challenged by practical and sometimes complex issues such as:

• Unreliable or restricted access to PCs and IT infrastructure; resulting in a lack of access to contemporary information; learning materials and social networks.
• Students experiencing uncertainty due to inadequate feedback on their performance resulting in unpreparedness; as well as insufficient monitoring, mentoring and tutoring (pastoral care) due to inadequate resources.

A potential solution explored is the ‘Assessment and Learning in Practical Settings’ (ALPS) programme, which employed mobile technologies. Common competencies, representing ‘readiness’ for professional practice were identified and developed into mobile assessment tools—which enable and encourage students to seek a spectrum of assessment opportunities such as from peers in the health and social care professions; from service users (patients); and self-reflection. The ‘ALPS’ assessments are completed are automatically captured online in e-portfolios; which in turn enable tutor and student review and commentary—in this way future practice of students are shaped and high-level scaffolding feasible. The University of Leeds, for example, has several hundred students in six different discipline/qualification cohorts.

Coulby, C; Laxton, J; Boomer, S & Davies, N. 2011. Mobile technology and assessment: A case study from the ALPS (Assessment and Learning in Practical Settings) programme, pp. 51-70. In N Pachler, C Pimmer & J Seipold (Eds). 2011. Work-based Mobile Learning — concepts and cases. Oxford: Peter Lang.

Work-integrated learning assessment methods and practices

Appropriate work-integrated learning assessment methods are a challenge. Apart from maintaining academic standards; relevance and consistency of assessment; deciding what should be assessed; who is responsible, where and when; and how assessment is supported and managed; are among the challenging issues.

A range of potential methods include reflective journals, portfolios, ePortfolios, workplace mentor/supervisors reports and workplace projects. Assessment should not only address skill acquisition and workplace competencies, but also the acquisition of 'soft skills' related to practical application of theory learned. Additionally graduate attributes; what learned about workplace culture and standards of the community of practice. The workplace mentor or employer’s supervisor should authenticate the evidence. Dunn, Schier and Fonseca (2012) identified the following key elements of an ePortfolio:

Dunn, L.; Schier, M. & Fonseca, L. 2012. An innovative multidisciplinary model for work placement assessment. Asia-Pacific Journal of Cooperative Education, 2012(3), 135-145.

Work-integrated learning as method of structured distance learning

The South African Ministry of Higher Education and Training (MHET) published the draft Policy Framework for the Provision of Distance Higher Education in the Government Gazette on 29 May 2012.

Based on the MHET Policy’s glossary (p. 4) definition of distance education work-integrated learning (WIL) can be regarded as a method of structured distance learning:

... a set of teaching and learning strategies (or educational methods) that can be used to overcome spatial and/or temporal separation between educators and students. However, it is not a single mode of delivery. It is a collection of methods for the provision of structured learning. It avoids the need for students to discover the curriculum by attending classes frequently and for long periods. Rather, it aims to create a quality learning environment using an appropriate combination of different media, tutorial support, peer group discussion, and practical sessions (emphasis added).

This is emphasised on page 8 of the draft MHET Policy that states: "Distance education proceeds from the belief that learning can be nurtured without necessarily requiring teachers and learners to be in the same place at the same time, and that the resource based nature of distance education allows for the possibility to achieve economies of scale" (emphasis added).

Page 8 of the draft MHET Policy also points out that distance education 'requires substantial up-front investment in curriculum design and materials development, including attention to issues of structure, pacing and meaningful formative assessment, as well as considerable investment in decentralised student support' (emphasis added), which is also true about WIL. Page 12 of the draft MHET Policy continues to state: 'Online communication allows students and academics to remain separated by space and time (although some forms of communication assume people congregating at a common time), but to sustain an ongoing dialogue. Online asynchronous discussion forums, for example, reflect an instance where the spatial separation between educator and learners is removed by the ‘virtual’ space of the Internet, but where there remains temporal separation' (emphasis added). The facilitation of reflection among students during and after stints of WIL has been emphasised.

Item 4.2.2 (h), pp. 22-23 of the draft MHET Policy giving the example of teacher education should enjoy appropriate attention:

Teacher education is a good example here: it remains in the lowest CESM funding category even though new accreditation requirements require investment in a robust teaching practice placement, mentoring and supervision system. While teaching practice has been funded as a full course at funding level 1, the issue now is to define it as an experiential learning component through the funding review process and thus fund it accordingly. Similar concerns arise regarding any programmes of study that require substantive work-integrated-learning and/or practical components and/or substantial online engagement between teachers and students in synchronous and asynchronous fora. Such requirements militate against the potential of achieving economies of scale.

It needs to be pointed out that the statement above does not mention the role of the teaching community of practice, which is a very important oversight. The collaborative notion in Cooperative and Work-integrated Education is not mentioned in the draft policy. The establishment of a supportive community of practice is crucial with regard to curriculum design, especially if work-integrated learning is decided upon as most appropriate method.

Practitioners, serving as mentors for learners, are the third key players in programmes that include work-integrated learning as preferred method. The up-front investment in for quality work-integrated learning includes appropriate materials development that would structure and pace learning as well as empower workplace mentors to offer meaningful formative assessment. This is where the investment of decentralised student support of quality work-integrated learning should focus. (Page 8 of the draft MHET Policy)

Although improvement of the quality of distance education is mentioned in section 4.3 (p. 23); concern about quality and of “inadequate practical work or exposure to work integrated learning” is highlighted.

Republic of South Africa. 2012. Department of Higher Education and Training. Draft Policy Framework for the Provision of Distance Higher Education in South African Universities. Retrieved online from: http://www.dhet.gov.za/LinkClick.aspx?fileticket=ZgZKPb1fM38%3D&tabid=36

Work based learners increasingly supported online

Online learning environments are increasingly being used to provide learning materials and tutor support, says Tony Toole, Director of the E-college.ac Ltd, Swansea, UK. Work-based learners may submit evidence portfolios online for the assessment of their skills development. Furthermore, employers and training advisors or academics working in partnership and liaise with each other online. Internet has significantly improved the efficiency of work-based learning support and offers significant cost savings for both employers and institutions. This is especially of value to small to medium-sized enterprises (SMEs) because they can more flexibly engage in work-based learning and can fit learning around work commitments with the added advantage of not involving time and travelling costs.

Web 2.0 technologies is utilised to build online learning environments with functionality similar to established commercial learning management systems, but far less costly. Cloud computing resources further enables reduced costs compared to hosting services online. Various social media, for example, YouTube, Blogs and Wikis further enables the sharing of resources. The Open Educational Resources (OER) movement resulted in fast volumes of educational materials being freely available online. Increasingly educationally relevant online work-based learning materials are being sourced from non-educational sources. The increase in personally owned mobile and wireless technologies further contributes to a win-win situation for all stakeholders in education. Blended and distance learning options are therefore becoming the preferred choice of learning.

Toole, T. 2011. Social media: key tools for the future of work-based learning. Development and Learning in Organizations, 25 (5), 31-34. Accessed online from Emerald Group Publishing Limited, ISSN 1477-7282.

Log as in shingles cut from wooden logs used for books

Keeping a log book is a common activity in learning from work experience. In many cases it is used as evidence of experience acquired and entries ratified by workplace supervisors. But where does the term log book originate from? I was pleasantly surprised today to stumble over the term among navy terminology: “In the early days of sailing ships, the ship's records were written on shingles cut from logs. These shingles were hinged and opened like a book. The record was called the ‘log book.’ Later on, when paper was readily available and bound into books, the record maintained it name.” Also of interest are terms such as:

• Hunky-Dory
• Long Shot
• Port Holes
• Taken Aback
• Took the Wind Out of His Sails

Development through assessment centre simulations

The three common uses of assessment centres (slide 25) include (a) selection; (b) diagnosis; and (c) development. While reading about simulations I recalled my own experiences with assessment centres. A quick search confirmed my expectations.

Journal of Cooperative Education and Internships now open access

JCEI access for individuals is now open to the public at no charge! Vol 45 Issue 01 has just been published:

Topic	Page
International centre of social science education and research in The Hague, The Netherlands.	5
Development of Learning Outcomes Assessment Methods for Co-operative Education Programs	11
Where’s WIL? Including work-integrated learning in descriptions of what it is that academics do	34
Comparative Curriculum Analysis of Mechanical Engineering Technology Programs at Purdue University and the National Taipei University of Technology	45
Mentoring Latinos in STEM: Transforming Struggling Co-op Students into Savvy Professionals	53
Developing an Online Learning Community for Engineering, Cooperative-Education Students: A Design-Based Research Study	67
Professionalising Accounting Education – The WIL Experience	80
Responding to Rapid Change in Higher Education: Enabling University Departments Responsible for Work Related Programs Through Boundary Spanning	94

JCEI is offered through EBSCO's library subscription for institutions. Request your university library to purchase subscription.

Where to start procedures for cooperative and work-integrated education?

At collaboration! CWIE is about cooperative education and about integrating work in education. An educational institution cannot offer CWIE programmes in isolation. Partnerships/collaboration is essential.

Internationally, it is considered essential to work according to unambiguous standard operating procedures (SOPs) to maintain quality. A SOP is a “written procedure prescribed for repetitive use as a practice, in accordance with agreed upon specifications aimed at obtaining a desired outcome”.

Why integrate work experience in a learning programme? The South African Qualifications Authority (SAQA) defines a learning programme as “the sequential learning activities, associated with curriculum implementation, leading to the achievement of a particular qualification or part qualification”. The decision to include work experience in a learning programme is a curriculum decision—where curriculum “refers to all of the teaching and learning opportunities that take place in learning institutions including:

• purpose and values of the learning
• learning outcomes
• content, activities, methods, media
• teaching, learning strategies
• forms of assessment
• evaluation of delivery, moderation”

If workplace experience is considered necessary it is included. But who decided? Stakeholder representation in the decision making is essential for determining fitness of purpose. However, unless the perceived relevant workplace experience can be adequately replicated within the teaching delivery, partnerships with sufficient stakeholder hosts are an undeniable necessity.

The starting point for any qualification therefore is adequate stakeholder representation. If during such stakeholder deliberations about the curriculum of the qualification indicate that inclusion workplace learning is regarded necessary, then shared responsibility and stakeholder partnerships become a condition of the learning programme. This is how cooperative work-integrated education become part of the teaching strategy.

It is further important that the educational institution nurture and expand the stakeholder involvement and establish sufficient partnerships to have sufficient hosting opportunities for the credit-bearing curriculum part work experience.

The nature of the work experience; the learning content thereof; how the learning would be acquired; the methods of accumulation of evidence of such learning; and the assessment thereof forms part of the teaching and learning development that flows from the curriculum decision.

The methods of maintaining communication with stakeholders; the identification of students in need of placement; the facilitation of the selection by organisations as to which student/s will be hosted; the preparation of students for such selection; the placement procedures and contracting with host organisations; the monitoring of the required workplace experience and monitoring of the mentoring of the learning; the monitoring of formative assessment and the summative assessment of accomplishment; would all form part of the SOPs.

Curriculum design decision responsive to stakeholder needs

The inclusion of work-integrated learning in the curriculum of an educational programme is a design decision, in response to stakeholder needs.

Stakeholder undertaking with regard to supporting work-integrated learning is crucial for the success thereof.

An educational institution that fails to grasp these two points is likely to struggle, if not fail, to make work-integrated learning work.

An educational institution that disregards continued stakeholder relationships underestimates the importance thereof.

An educational institute must ensure adequate representation of stakeholders in the design and/or revision of the curriculum. Such a curriculum-design collaboration must conclude which parts (if any) of the curriculum would be best achieved through real-life experience, rather than any other learning-mode. The occupational-field representation must consider it feasible to provide the relevant real-life learning contexts and mentors and undertake to make it happen.

The curriculum design and/or revision must never be an isolated event. An advisory structure, with adequate representation of stakeholders, must guide the offering of the programme and in particular the various aspects that enhance work-integrated learning, such as:

• Learning materials that guide the relevant real-life experiences would bring about the relevant learning.
• Materials to orientate students upon entry into work-integrated learning and guidelines for work-place mentors.
• Materials that would facilitate reflection on work experiences in order to bring about learning.
• Specifications and guidelines with regard to learning evidence required for summative assessment of work-integrated learning.
• Guidelines for the monitoring of the progress of students during work-integrated learning and formative assessment purposes.

Three key dimensions of work-integrated learning

When work-integrated learning is considered a necessary part of the curriculum of a qualification, it is important to systemically link three key dimensions, namely: teaching, marketing & counselling.

Two of the Ps of marketing are important considerations. Firstly, product—educational institutions are obliged to consult the relevant occupational field in the curriculum design of a qualification. Adequate representation of stake holders (including statutory, professional and/or vocational bodies) is important, not only for relevancy, but also for buy-in and support—an institution cannot offer a qualification that includes work-integrated learning, without sufficient host organisations willing to take students for real-life experiences in accordance with the curriculum outcomes. It follows logically that if there is good support within the occupational field that work-integrated learning is an important aspect of the curriculum, then less marketing (place, another P) efforts would be required to secure sufficient host organisations for students to undergo experience. It further follows logically that to ensure continuous relevance of the qualification and the support of the occupational field, continuous consultation is necessary—the formation and regular gathering of an advisory committee is therefore advocated. Advocates of academic freedom resist the idea of consumerism; however, public higher education institutions are accountable to its public and the state. Institutions might as well reap the benefit of sound marketing—satisfying the needs of the occupational/professional field the qualification intends to serve.

Teaching involves many aspects. It is among others very important to design a curriculum of a qualification as a whole and develop the learning of each part of the curriculum with the overall outcome in mind. Work-integrated learning as part of the overall curriculum requires special attention because students are generally removed from the institution—the learning takes place in a environment that academia do not have control over, yet the institution is ultimately responsible to ensure the learning outcomes were achieved. Orientation of students to the desired learning and a learning design that would bring about the desired learning is important. The workplace mentor needs to be given guidance in order to facilitate the relevant learning. The learning design includes the design of the appropriate learning evidence that will serve as substantiation that the student mastered the required learning outcomes. Formative assessment and summative evaluation of work-integrated learning also forms part of the teaching. In the South African context the Higher Education Quality Committee (HEQC) of the Council for Higher Education (CHE) determined a number of criteria towards good practice with regard to work-integrated learning.

Students come from various contexts; some are better prepared for entry into the world of work than other. Those underprepared students need counselling-type interventions to aid their development. Often higher education institutions offer a range of career and counselling services, such as CV writing, business writing, life skills, assertiveness, communication skills, etc. In the South African context the Department of Higher Education & Training gazetted that institutions that offer qualifications that include work-integrated learning are obliged to make sure that students are placed for such learning. It is advisable that those institutional support staff responsible to secure willing host organisations work together with the career and counselling staff in preparing students for the selection for work-integrated learning placements. It should be evident that the marketing aspect outlined in the first paragraph is crucial to the gazetted placement obligation.

Revised 17 August 2010: What I omitted, in error, from the above is the organisational perspective of talent growth through programmes that include work-integrated learning. The entire dimension of succession planning, people/talent development and the facilitation of organisational learning.

Analysis of experiential narrative instead of portfolios

It has often been said that it is naive to assume that students would actually learn from an in vivo (real-life) setting what they need to—i.e. what has been specified in the curriculum. It is my experience that students do not necessary experience, despite prior arrangements, what they should from a work placement. This is because of various reasons, among others work pressures. Besides the learning environmental factors, the student's own attitude may also restrict the student's learning.

In vivo learning educators/coordinators/work-integrated learning lecturers have tried many ways to bring about the desired learning of students, for example:

• Conducting on-campus orientation prior to placement of students in the in vivo learning setting (going for experiential/work-integrated learning)
• Providing students with work-integrated learning guidelines, detailing the relevant learning outcomes students must achieve and specifying what evidence of learning is required (this is especially important in the case of distance education institutions), for example, what the portfolio of evidence or work-project report should include
• Providing mentors (the work-place learning supervisors) with guidelines detailing what the students should be exposed to, their roles a learning mentors and what evidence of learning is expected from students
• Monitoring the learning and progress of students during in vivo/work-integrated learning
• Facilitate debriefing (or reflection) of students on return from in vivo/work-integrated learning stints away from campus
• Providing students with log books (to capture their experience and have each entry ratified) or with journals to capture their learning
• Require of students to present their portfolios to a panel that would interrogate their mastery of the specified learning outcomes

What about making use of story-telling and narrative analysis?

The in vivo/work-integrated learning guide would include the learning outcomes to be accomplished and suggested in vivo (real-life) settings.

The guide would further include (a) examples of daily journaling and/or recording of experiences, observations and realisations; (b) some indicators about story telling; and (c) a narrative analysis model—such as Lieblich, Tuval-Mashiach and Zilber (1998)—with basic explanations.

The instruction would be simple:

1. when the student is of opinion that she/he has mastered the specified learning outcomes she/he must write or tell the story about the learning experiences. If told, the student must transcribe the audio recording to text and edit it for readability
2. to do an analysis of her/his own narrative/experience-story (making use of the model in the guide)
3. illustrate/explain/qualify by linking up the analysis of the narrative with the specified learning outcomes to substantiate why she/he is of opinion the outcomes were mastered/accomplished

What do blog readers think? Could such an approach work? If readers think not, why? What are the potential pitfalls?

Feedback from a friend:

"Hi, Thomas.

Taking a rather oblique look at the problem, I'd ask, "What is it about the situation that mitigates against the desired outcome?"  Assuming the student has been given all the essential input (as you've listed) he knows what is wanted of him.   OK.  He's probably in a real-life workplace for the first time; it's and entirely new and perhaps stressful experience.  He's encountering the sort of people, rules, procedures and environment he's never experienced before.  Pressures will be on him to perform a job.   He may be earning a salary for the first time - which will be a distraction.  It will be during what is normally a holiday period, and his friends may be on the beach, or whatever. -- Just some ideas, based on observation of "Vac Students" in the Chemical industry."

Learner-managed learning through work (LMLTW)

Boud and Symes (in Choy) stated nearly a decade ago

When learning challenges are created from the exigencies of work there is often on pre-existing map that represents the territory of knowledge being explored.

Exigencies are pressing or urgent situations, requiring immediate effort or action; it is intrinsic to circumstances; cases that demand prompt remedy; a cricis; a contingency; a predicament; a fix; a pinch.

• When organisations sponsor the education of staff, it is done with growing human resource capacity in mind and therefore only expected to be aligned with business goals. Something the university for industry succeeded in. It requires a paradigm shift from traditional higher education. It requires a profound shift in the academic approach.
• Service learning in the true sense of the concept cannot be prescriptive. Community needs are examples of exigencies.
• The advocates of problem-based/orientated curriculum design, argue for focussing on what challenges graduates would face and developing ability to and knowhow where and how to find the information and/or solutions to address the exigencies they would face as graduate.

Choy (2009: 143) states that "the origins of knowledge and their boundaries in a work-based learning context are a lot more nebulous compared to fixed disciplines of academia", which is a matter (I believe) a lot of academia have not grasped yet.

The proponents of organisation-centred curriculums (OCC) founded and agreed on a framework around three principles (Choy 2009: 144):

1. partnerships between the educational institution and the organisation/s
2. learning activities are based on the needs of the organisation/s and the worker-learners, therefore linked to work projects
3. assessment and evaluation are in terms of both the agreed work outcomes and the academic learning objectives - both these inform the standards and measures

Choy reports on a case study in this regard and use terms such as a 'boutique curriculum' which is the result of interactions between disciplinary area, professional practice and workplace goals. Learners are obviously interested in immediate application and transfer of newly acquired knowledge and skills. Choy make reference to the Structure of Observed Learning Outcomes (SOLO) model and Hierarchy of Learning Outcomes (HLO) to accomplish constructive assessment. Another important notion mentioned is the compilation of resource folders (comprising tools and uasable materials acquired during the course of the learning process). Some real stimulating stuff here.

References:

Choy, S. 2009. Teaching and assessment for an organisation-centred curriculum. Journal of Workplace Learning, 21(2), 143-155.

Assessing in vivo learning outcomes and competencies or not?

Alderman and Milne (2005) observe that work-based learning is now a significant part of numerous higher education qualifications, however, the monitoring (among others) thereof is subject to much debate. “Work-based learning (WBL) is undertaken in a wide variety of higher education contexts and is increasingly viewed as a valuable, and increasingly essential, component of both the undergraduate and postgraduate student learning experience. However, the development of rigorous pedagogies to underpin WBL and its assessment is still embryonic.” (Brodie & Irving 2008). The purpose of this post is an attempt to add a little to the existing body of knowledge.

 

Some qualifications, in the light of their particular outcomes, are designed to incorporate periods of required work-experience (real-life learning) that integrate with the academic study. Where Work-integrated Learning (WIL) is a structured part of such a qualification the volume (number of academic credits) of learning allocated thereto, should be appropriate to the purpose of the qualification and to the cognitive demands of the learning outcome and assessment criteria contained in the appropriate level descriptors. Following this expanded version of a paragraph (p. 9) that appeared in the South African Higher Education Qualifications Framework (HEQF) as policy in terms of the Higher Education Act, gazetted by the Department of Education (DoE 2007), it is stated:

It is the responsibility of institutions, which offer programmes requiring WIL credits to place students into WIL programmes. Such programmes must be appropriately structured, properly supervised and assessed.

 

Two 2004 publications of the Higher Education Quality Committee (HEQC), a permanent structure of the South African Council on Higher Education (CHE), detail the last sentence of the quotation above. The CHE is an independent statutory body, which is responsible for advising the South African Minister of Education about higher education policy matters. The HEQC, led by an Executive Director, has executive responsibility for quality promotion and quality assurance of all higher education institutions. The two sets of HEQC (2004) criteria, namely two for Institutional Audits and for Programme Accreditation, therefore serve as important imperatives. A synopsis thereof includes:

·         Effective management and coordination, clear delineation of responsibilities and adequate provision of resources

·         Structured learning programme to accomplish the outcomes and learning agreements

·         Communication between host organisations, the institution and students

·         Mentoring system (supervision in the work place) that enables the student to recognise strengths and weaknesses, develop abilities, and to gain knowledge of work practices

·         Monitoring and recording systems to regularly and systematically assess the progress and learning of students in the workplace

 

An assessment workshop of the Geraldine Rockefeller Dodge Foundation (Grant 2007:1) generated a nine significant concepts and principles regarding assessment, which ties in with the mentioned short course, namely:

1.       “The primary purpose of assessment is to improve performance, not audit it”, which ties up with the mentoring imperative of the HEQC.

2.       “Good assessment requires being clear about mission and goals, the standards to which you aspire, and the criteria by which you would measure success.

3.       Therefore, it is about MEASURING WHAT MATTERS. (If you assess what you value, others will value what you assess.)

4.       And, necessarily, it becomes about PLANNING BACKWARDS.

5.       Assessment that improves performance involves FEEDBACK.

6.       One tool for getting useful feedback on what matters most is the RUBRIC.

7.       Good assessment requires a variety of measures, data, and feedback.

8.       Good assessment is ongoing. It is about continuous improvement. And unless we designate and protect the TIME to do this work, it will not happen.

9.       Done collectively, assessment builds community.”

 

Astin, Banta, Cross, El-Khawa, Ewell, Hutchings, Marchese, McClenney, Mentkowski, Miller, Moran and Wright (s.a.) support and supplement the nine principles above: “…it begins with educational values; …is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time; …works best when the programs it seeks to improve have clear, explicitly stated purposes; …requires attention to outcomes but also and equally to the experiences that lead to those outcomes; …works best when it is ongoing not episodic; …fosters wider improvement when representatives from across the educational community are involved; …makes a difference when it begins with issues of use and illuminates questions that people really care about; …is most likely to lead to improvement when it is part of a larger set of conditions that promote change; and through assessment, educators meet responsibilities to students and to the public.”

 

Burns (2008) remarks that assessment of the learning of students in clinical conditions; is unfortunately not always easy. She mentions patient participation in assessment of interpersonal skills, but expresses reservations about the ability to assess other competence. The Making Practice Based Learning Work (s.a.) website aims to enhance the quality of student experiences whilst on practice, i.e. work-based placements. This website is the product of a collaborative project involving staff from Bournemouth, Northumbria and Ulster universities as result of a grant from the Department of Employment and Learning (Northern Ireland) and the Higher Education Funding Council for England, to make practitioners more effective in and promoting the quality of practice based learning. In one of the many resources of this website, Allin and Turnock (s.a.) point out that assessment should be based on criteria that are known to the student; and also that assessment should be continuous—final judgement should never be on a single incident. Gray (in Allin & Turnock s.a.) observes that the contemporary focus of work-based learning is on competency standards, also called occupational and employment-related standards. Placement or work-based learning outcomes are therefore defined by means of outcomes to be achieved by students. Assessment, therefore take on a problem-centred approach, rather than knowledge-orientation. Demonstrating competency is about proving attainment of skills, not just writing about, and assessment becomes an integral part of the learning process. Bortot, Culberson, Modak, Becan-McBride and Niemand (2004) also emphasise that students must master defined competencies.

 

In contrast to the emphasis on outcomes and competencies, Bates (2003) argues that specific behavioural objectives cannot be specified in advance, because the moments of challenge cannot be predicted. Peterson (2004: 60) described such learning opportunities as “ambiguous, messy, and unclear” and observes that students “often do unexpected and serendipitous things”. Clark (2006:584) emphasises, in addition to “scientific dimensions of practice”, “technical knowledge and skills of the profession”, also “the so-called artistic … the ability to grapple with the gray or indeterminate areas of practice where moral ambiguity, value conflicts, and ethical dilemmas are commonplace”. Raths (in Bates 2003) states that the work-based learning (WBL) curriculum aught to be based on propositions about learning rather than specifically prescribed objectives. Bates (2003:303) found that a substantial body of literature address curriculum design and expected learning outcomes, but she did not find studies analysing “how students make sense” of the workplace experience. Baxter-Magolda (in Bates 2003) uses the term ‘self-authorship’ for the process that takes place as students incorporate their newfound learnings into their self-concept—“students construct knowledge as they construct ideas [that] they form about themselves” (p. 322). The WBL journals of students indicate that significant things happen unpredictably during WBL and that each placement entails different challenges. Gibbs & Panayiotis (2004) to some extent echo this by arguing that the assessment practices of higher education aught not enframe students, but rather liberate them by a notion of ‘letting learn’. Clark (2006:581) phrases the notion well by saying “learning is a continuous process grounded in experience, not an outcome”.

 

Ulmer (2001:68) cautioned about ‘patterning’ instead of learning—oversimplified it results in “Give me an example of how to think and do, and I’ll think and do like you.” Ulmer (2001) argues for problem-based learning—that is “learning that results from the process of working toward the understanding or resolution of a problem” (Barrows & Tamblyn in Peterson 2004)—and self-grading to achieve formative assessment. Ulmer emphasises that it is important that students know how they are learning; that they know if they are thinking critically; and that they can reflect on their own patterns of thought. Buchy & Quinlan (2000) made monitoring a participatory activity through a self-assessment scoring matrix. It develops the reflective thinking of students, as well as critical awareness of their own learning. Morton, Anderson, Frame, Moyes and Cameron (2006) found a stark contrast between the self-assessment of students and objective performance-based assessment of medical procedural competency.

 

It seems that there are two camps, one emphasising competencies and another that believes it limits. There appear to be a correlation between the schools of thought and the nature of the field of study, some do require specified competence, others are more open and some both.

 

REFERENCES

Alderman, B. & Milne, P. 2005. A model for work-based learning. Metuchen, NJ: Scarecrow Pr.

Allin, L. & Turnock, C. s.a. Helping you understand assessment [online]. Making Practice Based Learning Work. Available on the Internet at: http://www.practicebasedlearning.org/students/docs/Assessment%20in%20the%20Work%20Place%20for%20students.doc

Astin, A.W.; Banta, T.W.; Cross, K.P.; El-Khawa, E.; Ewell, P.T.; Hutchings, P.; Marchese, T. J.; McClenney, K.M.; Mentkowski, M.; Miller, M.A.; Moran, E.T.; Wright, B.D. s.a. Principles of Good Practice for Assessing Student Learning [online]. Mercer University. Available on the Internet at: http://www2.mercer.edu/Assessment/Principles.htm

Bates, M. (2003) The assessment of work integrated learning: symptoms of personal change. Journal of Criminal Justice Education, 14(2), 303-326.

Bortot, A.T.; Culberson, J.W.; Modak, I.; Becan-McBride, K. & Nieman, L.Z. 2004. Assessment of competencies in physicians-in-training through the delivery of a community-based health curriculum using distance learning. Medical Teacher, 26(7), 615-620.

Brodie, P. & Irving, K. 2007. Assessment in work-based learning: investigating a pedagogical approach to enhance student learning. Assessment & Evaluation in Higher Education, 32(1), 11-19.

Buchy, M. & Quinlan, K.M. 2000. Adapting the scoring matrix: a case study of adapting disciplinary tools for learning centred evaluation. Assessment & Evaluation in Higher Education, 25(1), 81-91.

Burns, D. 2008. A test of fitness: Dianne Burns asks if the profession should adopt new approaches to assessing student learning. Nursing Standard, 61(1), 23 April 2008, 22-23.

Clark, P.G. 2006. What would a theory of interprofessional education look like? Some suggestions for developing a theoretical framework for teamwork training. Journal of Interprofessional Care, 20(6), 577-589.

Dictionary.com [online] Available on the Internet at: http://dictionary.reference.com/browse/clinical

DoE see South Africa. Department of Education.

Gibbs, P. & Panayiotis, A. 2004. Accreditation of knowledge as being-in-the-world. Journal of Education and Work, 17(3), September 2004, 333-346.

Grant, D. 2007. Assessment Principles & Concepts [online]. Geraldine Rockefeller Dodge Foundation. Available on the Internet at: http://www.grdodge.org/learning/assessment/principles.htm

HEQC see South Africa. Council for Higher Education, Higher Education Quality Committee.

Making Practice Based Learning Work [online]. s.a. Available on the Internet at: http://www.practicebasedlearning.org/home.htm

Morton, J.; Anderson, L.; Frame, F.; Moyes, J. & Cameron, H. 2006. Back to the future: teaching medical students clinical procedures. Medical Teacher, 28(8), 723-728.

Peterson, T.O. 2004. Assessing performance in problem-based service-learning projects. New Directions for Teaching and Learning, 100, Winter 2004, 55-63.

South Africa. Department of Education. 2007. Higher Education Qualifications Framework. (Government Notice No 928), Government Gazette (No. 30353), 5 October 2007.

South Africa. Council for Higher Education, Higher Education Quality Committee. 2004. Criteria for Institutional Audits. Pretoria: Compress. Available on the Internet at: http://www.che.ac.za/documents/d000061/CHE_Institutional-Audit-Criteria_June2004.pdf

South Africa. Council for Higher Education, Higher Education Quality Committee. 2004. Criteria for Programme Accreditation. Pretoria. Available on the Internet at: http://www.che.ac.za/documents/d000084/CHE_accreditation_criteria_Nov2004.pdf

Starr-Glass, D. 2005. Metaphors and maps in evaluation. Assessment & Evaluation in Higher Education, 30(2), April 2005, 195-207.

Ulmer, M.B. 2001. Self-grading for formative assessment in problem-based learning. Academic Exchange Quarterly, 5(1), 68-71.

Knowledge - Competence - Performance - Action

The educational psychologist George Miller (in Norcini 2003) put forward a framework—illustrated in figure 1—for the assessment of clinical competence, i.e. proficiency based on actual observation of symptoms and treatment of disease in patients (Dictionary.com).

Miller (in Norcini 2003) differentiates between ‘action’, which is at the top of the framework, and lower levels, such as knowledge (at the lowest level), competence and performance. ‘Action’ is about what happens in practice. Norcini (2003) argues that the methods of work-based assessment should target the ‘action’ level. Care should be taken how data, for the basis of judgement, are collected. Data sources, such as diaries and portfolios, are recommended, but also observation.

 

 

Norcini, J.J. 2003. Work based assessment, ABC of learning and teaching in medicine. British Medical Journal, (326)7392, 5 April 2003, 653-5.