Evaluation

EVALUATION:
 

Evaluation is an integral aspect of Through the Looking Glass Project.

An evaluation plan was developed with a comprehensive plan of activities to be conducted to evaluate at all levels the success of this intervention. The activities are a combination of application of validated tools and processing of qualitative and quantitative data

The project adopted a participatory action research methodology and an independent evaluator from the University of Adelaide  Mr Paul Aylward and a research assistant Ms Margaret O'Niell were contracted to manage and coordinate the project evaluation. Qualitative and quantitative tools were applied to assess the results of the intervention with families participating in the project. Measurements using validated tools assessed both pre- and post- measures of attachment, parent sensitivity and emotional availability to confirm that project objectives were being met.

 The final evaluation report is now available and has been posted on the university web site.

Invest to Grow Final Evaluation Report
Through the Looking Glass – A Community Partnership in Parenting
Paul Aylward ‘Local Evaluator’ / Evaluation Manager,
Discipline of General Practice,
University of Adelaide and
Margaret O’Neill
Evaluation Assistant,
Lady Gowrie Adelaide,
South Australia.
May 2008

Paul Aylward and Margaret O’Neill have designed and conducted this
evaluation and produced this final draft report about the ‘Through the
Looking Glass’ (TtLG) Invest to Grow funded project. It draws on
information, opinions and advice provided by a variety of individuals
and organizations, including the Commonwealth of Australia. The
Commonwealth accepts no responsibility for the accuracy or
completeness of any material contained in this Interim Report.
Additionally, the Commonwealth disclaims all liability to any person in
respect of anything, and of the consequences of anything, done or
omitted to be done by any such person in reliance, whether wholly or
partially, upon any information presented in this publication.
CAUTION: Material in this publication is made available on the
understanding that the Commonwealth is not providing professional
advice. Before relying on any of the material in this publication, users
should obtain appropriate professional advice. Views and
recommendations which may also be included in this publication are
those of the authors Paul Aylward and Margaret O’Neil only, and do not
necessarily reflect the views of the Commonwealth, the Minister for
Families, Community Services and Indigenous Affairs, or the
Department of Families, Community Services and Indigenous Affairs or
indicate a commitment to a particular course of action.

1 To contact Paul Aylward:
paul.aylward@adelaide.edu.au
Ph: 0400 039 335

Acknowledgements
The authors would like to thank all the participants for the time they committed to
the evaluation including families, clinicians and child care staff from the participating
Centers. We would particularly like to acknowledge the Project team staff the
clinicians, co facilitators and primary care givers for their efforts in helping to collect
evaluation data, and the contributions from the management team at Lady Gowrie,
Pam Murphy and Kaye Colmer (including the provision of project materials for this
report). We would also like to acknowledge the guidance of the Reference Group in
the selection of suitable standardized tools used as part of this evaluation.
We would also like to acknowledge the contribution of the Commonwealth
Department of Family and Community Services and Indigenous Affairs for funding
this Project.

Acronyms used in this Report
CYH Child and Youth Health
CYWHS Child, Youth and Women’s Health Service
COS Circle of Security
COR Circle of Repair
EA Emotional Availability
HADS Hospital Anxiety Depression Scale
PCG Primary Caregiver
PSI Parenting Stress Index
TtLG Through the Looking Glass

1 SUMMARY
Background:
The Through the Looking Glass (TtLG) Project is a health, education and welfare
collaborative early intervention strategy that utilizes the existing infrastructure and
universality of five child care settings across Australia to intervene with families where
there is an identified compromised attachment relationship between the parent and
child/children.
The Project is designed to achieve specific outcomes for parents, children and child care
staff. The Lady Gowrie management team worked collaboratively with the local evaluator
to specify project objectives/outcomes. These were ordered hierarchically and an
evaluation plan designed.
The TtLG Project provides intensive psychosocial support, therapeutic intervention and
childcare as a package for high risk families in order to develop and support secure
attachment relationships between mother and child. The primary target group is
mothers of children aged 0-5 years. The participating families come from diverse
backgrounds but all exhibit multiple risk factors including anxiety, depression and social
isolation and many of the parents have reported early trauma in their own lives. There
are up to seven families recruited for each of the five Centres per Wave. There were six
Waves planned for the project each lasting around five months.
The TtLG project is based on Attachment Theory. The intervention draws from the
‘Circle of Security’ (COS) project model (Marvin et al., 2002) which assists parents and
child care staff to understand and integrate attachment theory into practice.
This report presents evaluation findings from the first five waves of the TtLG project.
Evaluation Methodology and Procedures:
This evaluation draws on Patton’s (1997) Utilization-focused Evaluation using
participatory action research procedures (Wadsworth, 1998: Sankaran et al 2001).
The evaluation also adheres to the tenets of ‘Realistic evaluation’ (Pawson and Tilley
1998). The evaluation, designed and managed by the external local evaluator has utilized
an Evaluation Assistant position based at Lady Gowrie Adelaide to enhance the
integration of evaluation procedures and on-going feedback into project practice.
Capacity training in evaluation for all project staff has been on-going, and formal
evaluation feedback has been facilitated through the Reference Group.
Evaluation data is obtained through applying a multi-facetted and methodologically
triangulated approach. The approach has been flexible in order to adapt to the evolving
Project and to minimize disruption to the busy workloads of those professionals
approached to participate.
In consultation with Lady Gowrie Management, the evaluators established ethically
appropriate systems to collect, compile and transfer confidential data from each site to a
central point for analysis. This included allocating unique case identifiers to each family,
child, site and Wave in order to link the various pre and post data sets. Clinicians were
engaged to help collect client data and were provided with client consent forms,
evaluation information and summary evaluation sheets for each to complete.
The evaluation uses a series pre and post Project measurement tools, surveys interviews
and observations to collect quantitative and qualitative data from mothers, children and
TtLG staff. The evaluation toolset is attached as Appendix D. The Reference Group was
requested to identify appropriated standardized instruments to measure a range of
psychological and behavioural dimensions related to the project aim. No one instrument
operationalised the multifaceted issues addressed and a suite of tests was subsequently
adopted. This necessitated the use of video recording, external assessment (by
professional assessors based in Sydney) and additional staff training.
Sustained impact for targeted families over the medium term were addressed through
follow-up surveys of all mothers from the first three waves three months after
completing the program. Sustained outcomes over a longer time period were addressed
through a follow-up survey of Wave 2 and Wave 3 families sixteen to eighteen months
after completing the project.
The experiences of project managers, site managers, project clinicians and project cofacilitators
have been addressed throughout the project through representations on the
Reference Group. However, as part of refining the service model the local evaluator
conducted a series of semi-structured (external) interviews with these professional staff
across all five project sites. To supplement these two focus groups of site staff who were
not directly involved with the project were also conducted to explore the extent to which
working practices across the child care centers had been influenced by the project.

Findings:
The Reference Group, Project management, childcare centre directors and TtLG staff are
working in partnership to develop and support the TtLG model. A range of suitable
partnering agencies have engaged with the Project and are committed to it. A number of
logistical issues have arisen in the implementation of the Project. The mechanisms for
identifying and addressing these issues have been established through the Reference
Group, on-going evaluation feedback, liaison between the sites and the Project manager
and through formal training and information exchange sessions which have been well
received.

The TtLG project has been very active in providing a range of capacity building activities
to staff across the five project sites. This has built capacity to adopt and deliver a
integrated primary care giving system, which in turn supports the TtLG families and
improves attachment outcomes. This has allowed the organisation to deliver better
services for targeted families and their children, (a national ‘Invest to Grow’ priority).
The project has an excellent client retention rate; 90% of the families recruited
completed the five month project (n=106). Formative evaluation has revealed that
mothers and fathers have been very positive about their experiences with the project and
these feelings continued after completion. Mothers enjoyed the sessions provided and
felt comfortable, relaxed and safe in the settings where they could freely explore their
parenting and attachment issues.

Given the multifaceted and holistic approach adopted in the project model, it is difficult
to identify the most important factors which facilitated improved impacts. More than
eight in ten mothers indicated that 80% of the strategies employed had helped them
(with six in ten indicating that 70% of strategies had helped them ‘a lot’) with regard to
understanding their child’s attachment needs. The combinations of group and
individual work with clinicians and reflections on the child/parent video films guided by
insights from attachment theory and the ‘circle of security’ have clearly contributed to
greater understanding of attachment. The childcare and primary care giving ethos of the
centers were also highly valued.

86% of the 106 mothers who completed the project indicated that it had helped them to
feel closer to their child, with nearly eight in ten indicating the project had helped them
to feel good about themselves as parents. 70% were more confident to look for other
services and supports for their family. Around nine in ten mothers indicated that they
had learnt more about parenting and attachment, were more confident to respond to
their child’s needs, were better able to cope as a parent, felt closer to their child and
acquired understanding of their child’s attachment and exploration needs. 88% of
mothers noted lasting positive changes in themselves since completing the project. All of
the mothers surveyed continued to apply learning and skills acquired through the project
16-18 months after completing it; mothers reported sustained benefits for their
parenting practice, well-being and family functioning. Around eight in ten mothers
formed supportive friendships during the project with over half of the mothers engaged
maintaining friendships three months after project completion. Whilst this reduced over
time, 28% of mothers indicated they had retained friendships 16-18 months after
completing the project.

The above findings have been supported through accounts of professional stakeholders,
and are further supported by the applied pre and post standardized tools. Psychological
and behavioural improvements were found to be statistically significant in nine of the
eleven dimensions measured, with large effect sizes found for reductions in depression,
anxiety and stress, and improvements in the child’s wellbeing and involvement
observation ratings. Over the duration of the project, the number of mothers
experiencing ‘moderate’ to ‘severe’ anxiety and depression (scoring between 11-21 on the
HADS), more than halved; from 52 to 25 (anxiety) and from 37 to 13 (depression).
Conversely the numbers acquiring a ‘normal’ score more than doubled for anxiety (from
21 in the pre measure to 44 in the post) and was 60% higher for depression (from 42 to
67 respectively).

The project has improved parent competence and style and improved family functioning.
Parents have increased their knowledge competence and awareness to overcome barriers
to attachment, are less stressed, depressed and anxious and better able to cope as
parents. Many report better parenting practices, better engagement with their children
and improved child behaviours which they attribute to the project. For many parents
these impacts have been sustained since leaving the project. These findings provide clear
evidence that the project is addressing the national ’Invest to Grow’ priority areas of:
‘Improved family functioning’, ‘Improved parent competence and style’ and ‘Improved
child social and emotional development’.

A range of issues have been identified with regard to optimising the implementation of
the model. Establishing a ‘primary care giving culture’ and broader understanding of
attachment theory require on-going training in a field known to have substantial staff
turnover. In several sites this task has been embraced as part of the role of the clinician.
Moreover, all the centres and staff engaged with the project are committed to the ongoing
implementation of primary care giving practice; this is ingrained in policy
developments at each centre.

Whilst fathers as a group have been engaged in several sites this has tended to be more
focussed on information giving activities. This has been more advanced in the longer
established Thebarton site where group sessions have included video activities. The
logistics of assembling fathers at convenient times have been prohibitive, and many of
the mothers do not have a male partner. However, staff have engaged with fathers
through their families as part of the PCG approach and this has become standard
practice for those involved with the project. Fathers engaged in formal group sessions
have benefited in terms of raised understanding of attachment and subsequent parental
improvements.

Few C&LD clients were engaged and this has been largely attributed to the demographics
of the site constituencies. Lady Gowrie Adelaide is currently conducting a separate study
investigating the applicability of the model with Indigenous communities. This study has
utilised funds from the project with the agreement of the funding body and is not part of
this evaluation.

Difficulties identified in the Interim Report concerning the adoption of multidisciplinary
team working have been addressed and appear to have largely been resolved
within each site. This has occurred as project staff have gained mutual appreciation of
the expertise each discipline has brought to the project. However, a number of areas of
potential refinement relating to this have been identified and are currently being
reviewed; these are detailed in Section 7.4 of this report1. These should be generally be
viewed as considerations for those seeking to implement the model rather than
stipulations as there will inevitably be contextual and staff differences in different site
locations.

Establishing a coherent set of working, reporting and accountability procedures across
the five engaged sites has proven to be highly problematic particularly as each is a
sovereign body. This has also been exacerbated by staff turnover and geographical
distance, notably with the Perth site which ended its involvement with the project after
the fifth wave. However, all sites (including Perth) have expressed strong wishes to
continue with the project in some form.

In the light of the evidence presented through this evaluation, there is an overwhelming
case to perpetuate the project in order to build on the investment and continue to
provide an intervention which has clear multiple positive impacts and sustainable
benefits for Australian families. Whilst there are areas of the service model which may be
subject to on-going context specific revision, the project demonstrates its flexibility to
adapt to and be adopted by different child center practices and contexts and generate a
range of successful and profound outcomes for service providers and their clients.
However, the need to secure funding for the Clinician and co-facilitator roles and to
support the provision of child care for project clients is crucially important to the
functioning of the project.

 

 2 PROJECT DESCRIPTION

The Through the Looking Glass (TtLG) Project is an attachment focused parenting
Project based at Lady Gowrie Child Centre Adelaide. TtLG began as a pilot Project in
2002 as a partnership project between Child and Youth Health (CYH) and Lady Gowrie
Child Centre with Commonwealth funding. In 2003 the original pilot was extended to
2004 with funding from the South Australian Department for Education and Children’s
Services (DECS).
A successful grant application in 2005 secured further funding from the Commonwealth
Government’s Stronger Families and Communities, Invest to Grow Strategy to expand
the Project across centres within Adelaide and interstate over a 3 year period. The
current TtLG Project involves 5 centres, 3 sites in metropolitan Adelaide and 2 interstate
sites, Brisbane and Perth:
1. Lady Gowrie Child Centre, Thebarton, Adelaide;
2. il nido Child Care Centre, Salisbury, Adelaide;
3. Salisbury Highway Child Care Centre, Salisbury Downs, Adelaide;
4. Lady Gowrie Child Centre, Brisbane;
5. Lady Gowrie Child Centre, Perth.

The TtLG Project is a 6 month intervention being implemented in 6 waves across all 5
childcare sites. Each site was limited to a maximum of seven TtLG families per Wave.
The Project commenced in July 2005 and is due for completion in June 2008. The first
five waves have been completed and Wave 6 is currently underway.

2.1 Project Focus:
The project differs from many more traditionally skill based parenting Projects by
focusing on the development of the attachment relationship between parent and child.
The project is a health, education and welfare collaborative early intervention strategy
that utilizes the existing infrastructure and universality of child care settings to intervene
with families where there is an identified compromised attachment relationship between
the parent and child/children. An innovative aspect of the project is the provision of up
to two days child care per week to participating families. This acknowledges the
importance of providing support to families to enable the development of quality
parenting. It also recognizes that secure attachment relationships between children and
their parents can be supported by child care staff.

2.2 Project Outcomes:
The Project is designed to achieve specific outcomes for parents, children and child care
staff. The Lady Gowrie management team worked collaboratively with the local evaluator
to specify project objectives/outcomes. These were ordered hierarchically and an
evaluation plan designed (see: Appendix A and Section 5 of this report).

2.3 The TtLG Project model
The TtLG Project provides intensive psychosocial support, therapeutic intervention and
childcare as a package for high risk families in order to develop and support secure
attachment relationships between mother and child. The primary target group is
mothers of children aged 0-5 years. The participating families come from diverse
backgrounds but all exhibit multiple risk factors including anxiety, depression and social
isolation and many of the parents have reported early trauma in their own lives.

2.4 Client Recruitment Procedures
Families were recruited to the Project through a variety of channels including from
within the service childcare centre and ‘self referral’. The majority of referrals were
obtained utilizing local linkages to a range of agencies including:
· Child Health Services;
· General Practice;
· Infant Mental Health Services;
· Early Childhood Education / Care;
· Child Protection Agencies;
· Local Church Agencies;
· Out Reach Projects by Non Government Organizations at the local level;
· Allied Health / Social Work / Psychology Departments of Major Children's
Hospitals;
· Children's Mental Health Services;
· Community / Neighborhood Houses;
· Community Health Services;
· Women's Health Services;
· Family Support Agencies;
· Children s Centres.
Agencies were provided with information materials and referral forms and the potential
recruits identified were subsequently offered an initial assessment at the nearest TtLG
childcare centre. Full details of these procedures are provided in the Project Manual (see:
Appendix C for the Manual contents).

2.5 The TtLG Intervention
Each participating childcare centre employs a clinician to work with families in
partnership with the childcare staff. The clinicians come from social work or psychology
backgrounds.
The TtLG intervention is multi faceted and incorporates:
· Provision of up to 2 days child care per week. The child care gap is paid for by
the project making the child care free to those families on maximum Child Care
Benefit and at a reduced cost for others.

· Primary care giving. The primary care giving (PCG) model of childcare provides
a secure base for each child by ensuring each child has a ‘special person’ and each
parent has a primary contact.·

Intensive 1:1 individual work with the clinician. To address individual challenges
and unresolved issues all families in the Project work with the clinician for
individual family work/counseling and support which is delivered at the child
care centre or through home visiting.

· An 18 session weekly group Project. A group Project is conducted for 2 hours
each week for the mothers whilst the child/children are in care. The small group
size supports the establishment of a safe secure environment to share and explore
parenting experiences.
The weekly group component of the Project is facilitated by both the clinician and
a childcare co-facilitator. In the Project both the clinician and child care workers
work in partnership with each other and the family. The primary caregiver (child
care) has a vital role in supporting the parent to achieve their set goals. They
develop a significant relationship with the child and the parent and work closely
with the clinician to develop and enhance the attachment relationship.
The sessions are a mix of educative and therapeutic activities offering
information and resources which assist mothers to reflect on their relationships,
to understand the nature of healthy attachment and examine issues that may be
inhibiting their capacity to respond to the child’s needs.

· Video taping of parent child interactions for parent reflection. Video taping is a
key intervention tool in the Project. Parents can explore attachment relationship
needs by observation and reflection with the clinician both during individual
family work and also within the group setting.

· Partnerships between parents, workers and agencies. The clinician, parent and
primary caregiver (child care) meet on a regular basis to work together to meet
the parents goals. Referrals to other service providers and joint case conferencing
are regular practices.

· Learning stories. Child care primary caregivers develop with the child, stories
about their daily activities which communicate from the child to their parent
their relationships, learning and development within the child care setting.
Families are provided with stories which specifically report on their child in
relationships that nurture and support exploration.

· Staff Training and Professional Development. Building staff capacity to work
with vulnerable families and to apply attachment theory to their work.

· Specific father’s sessions. When appropriate, short group sessions are provided
for fathers which enable them to be involved in some of the activities which are
delivered to their partners as part of the 18 week Project.

The Project works directly with up to 7 families in each group and focuses on their
particular defenses that are directly impacting on their attachment relationship with
their child/children. The childcare provided and group processes facilitated play an
important role in providing a secure base for the parent enabling them to maximize their
exploration, reflections and considerations of their relationships with their children.

2.6 Integration of ‘Attachment Theory’
The TtLG project is based on Attachment Theory. The intervention draws from the
‘Circle of Security’ (COS) project model (Marvin et al., 2002) which assists parents and
child care staff to understand and integrate attachment theory into practice. The project
specifically utilizes the ‘COS’ graphic. This model provides an understanding of
children’s behaviour from an emotional needs perspective and has made attachment
concepts more accessible to parents and professionals working with them.

 

3 PROGRAM LOGIC
The program logic is demonstrated in the evaluation plan by clearly linking the project
components (overall goal and listed objectives, the strategies to achieve these, the
process indicators to address the strategies, the impact indicators to address the
objectives and the methods to collect data for these indicators). The components of the
evaluation are clearly and logically related. The objectives have been hierarchically
ordered (sometimes referred to as an ‘outcomes hierarchy’) in logical fashion. Moreover
the strategies for each objective have also been logically sequenced.
There are a number of ways of explicating program logic. The one chosen here has been
popularized through the ‘Planning and Evaluation Wizard’ (PEW) co-authored by the
local evaluator and adopted for teaching evaluation in Public Health Honors and postgraduate
courses in at least five Australian Universities. The model used here is most
appropriate for participatory approaches as it is intuitively easy to understand and has
been applied through Primary Health Care Research and Evaluation Development
(PHCRED) in myriad community health and General Practice contexts. This approach
was also used by the local evaluator when awarded a ‘National Commendation for
Excellence in Evaluation’ by the Australasian Evaluation Society. The approach has been
used in the capacity building activities conducted with TtLG staff as part of this
evaluation and has been well received.
The Evaluation Plan Matrix is attached as Appendix A

 

4 LITERATURE REVIEW
The evaluation draws on Patton’s (1997) Utilization-focused Evaluation approach and
uses participatory action research procedures (Wadsworth, 1998: Sankaran et al 2001).
The evaluation also adheres to the tenets of ‘Realistic evaluation’ (Pawson and Tilley
1998). The design of the evaluation plan has been based on the PEW model3 and this
with the evaluation procedures adopted has been published elsewhere4.
4.1 References:
Abidin, R.R. (1995). Parenting Stress Index Professional Manual (3rd ed). PAR
Psychological Assessment Resources, Inc. Florida. www.parinc.com
Aylward, P. (2005). Evaluation for AOD Projects and Projects. In N. Skinner, A.M.
Roche, J. O'Connor, Y. Pollard, C. Todd (Eds.), Workforce Development TIPS
(Theory Into Practice Strategies): A Resource Kit for the Alcohol and Other
Drugs Field. National Centre for Education and Training on Addiction (NCETA),
Flinders University, Adelaide, Australia
Beebe, J. (2001). Rapid Assessment Process. Walnut Creek, CA: AltaMira.
Biringen, Z., et al. (1998). The Emotional Availability Scales, (3rd ed.), Attachment and
Human Development, 2, 256-270.
Handwerker, W.P. (2001). Quick Ethnography. Walnut Creek, CA: AltaMira.
Biringen, Z., et al. (2000). Emotional Availability Scales, (3rd ed.), Unpublished manual,
Department of Human Development and Family Studies, Colorado State
University, Fort Collins.
ICC/ESOMAR International Code of Marketing and Social Research Practice (2001).
http://www.esomar.nl/guidelines/ICC_ESOMARcode.htm
Laevers, F., Debruyckere, G., Silkens, K., & Snoeck, G. (2005). Observation of Well-Being
and Involvement in Babies and Toddlers. CEGO Publishers Leuven Belgium.
Laevers, F. (ed.) 2005, Well-being and Involvement in Care Settings. A Process-oriented
Self-evaluation Instrument (SiCs), Brussels, Kind&Gezin
Marvin, R., Cooper, G., Hoffman, K. & Powell, B. (2002). The Circle of Security™
Project: Attachment-based intervention with caregiver-pre-school child dyads.
Attachment & Human Development, Vol. 4, No 1, April 2002, 107-124.
Murrey, C., Aylward,P., et al (2001) Project Evaluation Wizard (PEW). SACHRU.
http://www.sachru.sa.gov.au/PEW/index.htm
Pawson, R., & Tilley, N. (1998). Realistic evaluation. London: Sage.
3 Murrey, C., .Aylward, P., et al (2001).
4 Aylward, P. (2005).
Quinn Patton, M. (1997). Utilization-focussed evaluation (3rd ed.)Thousand Oaks, CA:
Sage.
Sankaran, S., Dick, B., Passfield, R., & Swepson, P. (2001). Action learning and action
research. Lismore, NSW: Southern Cross University Press.
Wadsworth, Y. (1998). What is participatory action research? Action Research
International, Paper 2.
Winter, P. (2003). Curriculum for babies and toddlers: a critical evaluation of the first
phase (birth to age three) of the South Australian Curriculum, Standards and
Accountability Framework in selected childcare centres in South Australia.
Unpublished PhD dissertation. Magill: University of South Australia.
Zigmond, A.S., & Snaith R.P. (1983). The Hospital Anxiety and Depression Scale. Act
Psychiatr Scand, 67:361-70.
Snaith, R.P. (2003). The Hospital and Depression Scale. Health and Quality of Life
Outcomes 1:29 http://www.hglo.com/content/1/29 accessed 6/09/2005.

 

5 EVALUATION
5.1 Evaluation Methodology
The local evaluator liaised with The Lady Gowrie Child Centre Inc South Australia to
clarify the goal and objectives of the Project, identify a range of indicators for each stated
objective and compile an Evaluation Plan.
The evaluation goal and objectives are:
Goal:
To develop and pilot a model of collaborative early intervention and
prevention for targeted parents to improve secure attachment outcomes
for young children in five selected child centre sites across Australia.
Objectives:
1. To forge working and sustainable inter-sectoral partnerships across
Australia (childcare, health, education and consumer) overseeing and
informing the development and management of the Project.
2. Build capacity of participating Childcare Centers to develop and adopt a
sustainable integrated primary care-giver system
3a. To equip and empower a range of parents of young children with the
knowledge, awareness, confidence and skills to successfully overcome the
barriers to attachment
3b. To foster and nurture positive parent well-being outcomes
3c. To foster and nurture positive child well-being outcomes
4. Develop and enhance social support /friendship networks for the target
group
5. To develop and promote the uptake of a ‘best practice’ model for services
working with mothers and fathers and children around issues of
attachment

This evaluation adheres to the tenets of ‘Realistic evaluation’ (Pawson and Tilley 1998),
which highlights the importance of investigating the reasons why those individuals
targeted made or did not make the desired choices or engage in the desired behaviours
encouraged by the Project. This approach focuses on what worked, for whom, in what
context, and the mechanisms that made the Project work.
The evaluation has been collecting a broad range of triangulated data from identified
stakeholders engaged with the project including all clients (mothers) from each Wave.
This includes a range of qualitative approaches (in-depth interviews, focus groups, semistructured
telephone interviews and ‘rapid reconnaissance’), and quantitative
approaches (systematically collected demographic data, self-completion surveys, and the
application of pre and post standardized psychometric tools addressing a range of
psychosocial and behavioural dimensions). A follow-up telephone survey of all mothers
three months after completing the project has been included for the first three waves in
order to provide insights into their reflections on the project, further developments and
sustained medium term impacts. Longer term outcomes for clients were addressed
through a 16-18 month follow-up survey of Wave 2 and wave 3 mothers5.
This evaluation acknowledges the importance of both the well being of the project
clients, and the established (though potentially fragile) inter-relationships between them
and the services providers at participating sites. Given the potentially vulnerable client
base targeted by the Project, the need for an appropriate, respectful and sensitive
approach to the evaluation has been identified. However, the need to further explore the
contextual issues and personal experiences raised by participants is also important in
order to yield fuller understanding of the project, its operation and significance for those
connected with it. The local evaluation therefore purposely engaged with the project
team in partnership to facilitate data collection from clients recruited at each TtLG site.
An evaluation assistant was located at the Lady Gowrie Adelaide site to further engage
with staff, promote the integration and blending of evaluation procedures with those of
the TtLG project and to help coordinate the range of data collection procedures across
the five sites. Given the need to engage with the women who had completed the project,
the evaluation assistant selected was a mature female evaluator who was experienced in
conducting qualitative evaluation work with women in the health arena. The evaluation
 findings from the three-month follow-up were highly positive and demonstrated sustained
benefits over this period for parents and their children. These findings are included in this report.
Following a formal presentation of these findings to the Reference Group (as part of the action
research process), and given that the model of delivery had reached maturity by this stage, the
evaluator advocated redirecting evaluation resources to investigating longer term outcomes for
the targeted families. This was supported by the Reference Group. Wave 2 and Wave 3 families
were selected to enable the long term outcomes to be addressed within a ‘reportable’ time-frame.
assistant was overseen by the local evaluator who designed and managed the evaluation.
The local evaluator also conducted much of the primary data collection from professional
stakeholders (focus groups of PCGs and interviews with key staff in all project sites).
Given these factors, a ‘participatory’ approach to the evaluation research was been
adopted. This purposely enlists the collaboration of the Project team to help enact
suitable and sensitive data collection strategies in order to facilitate the gathering of
richer more authentic data from clients, whilst building capacity across the organisation.
The evaluation approach to clients has been guided by the ‘inside’ knowledge and
experiences of these stakeholders in a collaborative sense. Much of the data collection
has been integrated into clinical practice. In this sense, the methods applied whilst being
rigorous, were also flexible and sensitive to context.
Drawing on Michael Quinn Patton’s Utilization-focused Evaluation approach (1997), the
ethos of the evaluation harmonizes with that of the Project; The Lady Gowrie Child
Centre Inc are effectively engaged as evaluation partners in order to collect evaluation
data in contextually appropriate ways.
Additionally, the need to build capacity and sustainability not only of the Project but also
with regard to strengthening the evaluation component of future projects conducted by
the Lady Gowrie Child Centre Inc was well recognized by the Project staff and the local
evaluator. Formal and informal training in planning and conducting evaluation has been
provided to TtLG staff drawing on the expertise and experience of the local evaluator
who has developed and taught a range of research methods and evaluation graduate and
post-graduate courses over 18 years in the UK and Australia. The training process began
with the Evaluator liaising with the Lady Gowrie Child Centre Inc in the production of
the Evaluation Plan. The local evaluator has also delivered a series of formal task
orientated training sessions involving staff from the five project sites, with mentoring
being provided throughout the Project.
The evaluation embraces participatory action research procedures (see Wadsworth,
1998; Sankaran et al 2001), whereby findings are relayed back to the Project to facilitate
developmental improvement. This is accommodated through both informal partnership
channels and liaison with the Project team, and formally through reporting back to the
Reference Group throughout the Project. The local evaluator and evaluation assistant
were full partners in the Reference Group with ‘Project Evaluation’ was a standing item
on the agenda to facilitate feedback, reflection and action.
This evaluation fully conformed to NHMRC Guidelines, and the ICC/ESOMAR
International Code of Marketing and Social Research Practice (2001). Ethics approval
for the Evaluation of the TtLG Project was granted by the ‘Children Youth and Women’s
Health Service Human Research Ethics Committee’.

5.2 Evaluation Methods
Evaluation data is obtained through applying a multi-facetted and methodologically
triangulated approach. The approach has been flexible in order to adapt to the evolving
Project and to minimize disruption to the busy workloads of those professionals
approached to participate.
The evaluation uses a series pre and post Project measurement tools and surveys to
collect data from mothers, children and TtLG staff. The evaluation toolset is attached as
Appendix D.
In consultation with Lady Gowrie Management, the local evaluators established ethically
appropriate systems to collect, compile and transfer confidential data from each site to a
central point for analysis. This included allocating unique case identifiers to each family,
child, site and Wave in order to link the various pre and post data sets. Clinicians were
engaged to help collect client data and were provided with client consent forms,
evaluation information and summary evaluation sheets for each to complete (see:
Appendix D, 1-2).
The Reference Group was requested to identify appropriated standardized instruments
to measure a range of psychological and behavioural dimensions related to the project
aim. No one instrument operationalised the multifaceted issues addressed and a suite of
tests was subsequently adopted (see: Appendix D). This necessitated the use of video
recording, external assessment (by professional assessors based in Sydney) and
additional staff training. The tools selected for mothers and children where:
Mothers:
· The Hospital Anxiety and Depression Scale (HADS) measures change in a client’s
emotional state using anxiety and depression subscales (Zigmond & Snaith,
1983);
· Parenting Stress Index Short Form (PSI/SF) questionnaire measures stress in the
parent-child system (Abidin RR, 1995);
· Emotional Availability (EA) framework allows for measuring changes in the
parent-child relationship based on parent dimensions: sensitivity, structuring,
non-intrusiveness and non-hostility and child dimensions: child responsiveness
to parent and child involvement with parent (Biringen, Z., et al., 1998; Biringen,
Z., et al., 2000). Videotapes of mother and child interactions are assessed by
qualified professional EA scorers.
Children
· Children’s Wellbeing and Involvement Observations measure a child’s levels of
wellbeing and involvement while attending childcare, (Winter, P. 2003; Laevers,
F., et al., 2005). Observations are systematically recorded by childcare staff.
The local evaluation has also developed the following evaluation tools: (see: Appendix
D):
· Client demographic form based on National Evaluation Service Users
Questionnaire;
· Post Project questionnaires to measure mothers and fathers satisfaction and
experiences of the TtLG Project;
· Follow-up qualitative telephone interviews with mothers three months after
completion of the project to further explore reflections about the project and
identify sustained impacts;
· Email surveys for Reference Group members and TtLG co-facilitators
· Interview Schedules for Clinicians and Directors;
· Topic Guide: Qualitative Interview. TtLG Lady Gowrie Management;
· Longitudinal Follow-Up qualitative telephone interviews with mothers 15 months
after completion of project to explore sustained outcomes;
· Topic Guide: Focus Groups of Primary Care Givers;
· Semi Structured Interview schedule with Managers, Clinicians, Co-Facilitators.
At the Process Evaluation level the indicators and data collection methods (as specified
in the Evaluation Plan Matrix) assessed the implementation and activities of the TtLG
Project in relation to the Invest to Grow Project principles. Primary data is being
collected from all engaged parents, and professional stakeholders. This has been
complemented by observational data collected through ‘rapid reconnaissance6’
conducted at three ‘satellite’ childcare centre sites.
A range of impact and outcome indicators have been identified and included in the
Evaluation Plan Matrix. Whilst these specifically address the defined objectives for the
TtLG Project, they also address several of the national priority areas of the ‘Early
Childhood Invest to Grow Established and Developing Projects 2004-2008 Project
Guidelines’, namely:
· Supporting Families and Parents to develop strong parent/child
relationships, improve parenting competence and style, family capacity and
resources, and family functioning.
· Early Learning and Care to improve child social and emotional development.
· Child Friendly Communities that are inclusive of all families and cultures.
The outcomes specified for these priority areas are therefore highlighted by asterisks (*)
in the Evaluation Plan Matrix in order to make explicit the linkages to the national Invest
to Grow Project.

5.3 Evaluation Challenges and Changes
The TtLG Project was not implemented in synchronization across all 5 participating
childcare sites. Given the variety of evaluation data collected during each wave of the
TtLG Project (including pre and post Project measures for families), the coordination,
collection and compilation of evaluation data was a substantial challenge.
Much of the data collection was reliant on the cooperation and diligence of the clinicians
at each of the five sites. The need to plan and clarify data collection procedures was
crucial to ensuring this occurred efficiently particularly as the clinicians were all
employed on a part-time basis. Whilst these processes were addressed through training
and the provision of instructions and forms, the challenges of establishing the new
project in Wave 1, (and a degree of staff turnover later) inevitably led to some delays in
the collection and return of data to the evaluation assistant.
There were delays with the Reference Group decision making regarding an appropriate
attachment measure to be used in the evaluation. Having identified the Emotional
6 See Beebe, J. (2001) and Handwerker, W.P. (2001)
Availability tool, it became necessary to provide TtLG staff with training in videotaping
and editing skills to produce videotapes of mother-child interactions for EA assessments.
Logistical difficulties were engendered by the need to identify and engage professional
EA assessors to receive and rate videos of child parent interactions from all sites; the
only assessors qualified for this task were located in Sydney, NSW and the mechanisms
for channeling information from the five sites to the assessors and subsequently
transferring assessment scores back to the evaluator in Adelaide needed careful planning
and monitoring.
There were also some difficulties collecting the Children’s Wellbeing and Involvement
Observations scores. TtLG staff representatives received training in making the
observations at the July 2005 workshop and returned to their childcare centres where
they subsequently trained other staff members. However few observations were made in
Wave 1 and staff reported feeling a lack of confidence and skill in applying the scores.
Project management subsequently organised additional training and distributed a
training video to all sites. This led to an improvement in the number of observations
made. Observations subsequently improved. However other factors also impacted on
data collection including staff turnover of trained observers and the need to up skill
newly recruited staff members.
As a result of this, observation measurements have not been made of all children in the
TtLG Project. However as staff become more confident and skilled increasing numbers of
observations were made as outlined in Table 1.
Table 1: Application of the Wellbeing and Involvement Observation Tool by
Wave
See Full Report including tables by visiting the University of Adelaide website

Given the complexities of data gathering and staff turnover, the local evaluator has
delivered more training workshops than was originally envisaged in the evaluation
design. Moreover, the need for the evaluation to acquire a broader conceptual
understanding of how the project operates ‘in situ’ was identified; the local evaluation
therefore embraced an additional ‘quasi ethnographic’ method ‘rapid reconnaissance’ for
this purpose. Through observations and informal interviews with project staff at a
‘satellite’ site, further insights were gained regarding the practical application of the
project which complemented the ‘inside’ experiences of the evaluation assistant based at
the Gowrie Adelaide Thebarton Centre.
The introduction of staff representatives to the reference group mid-way through the
project provided the opportunity to acquire on-going information from their
perspectives more efficiently. This has been utilised by the evaluation using these
representatives as ‘key informants’.
The Evaluation Assistant needed to reduce her working hours mid-way through Wave 4
which precipitated the need for greater ‘hands on’ involvement form the Evaluation
Manager / ‘Local Evaluator’. This was facilitated through negotiation with the University
of Adelaide.

6 EVALUATION FINDINGS AND DISCUSSION
6.1 Objective 1: To forge working and sustainable inter-sectoral
partnerships across Australia (childcare, health, education and consumer)
overseeing and informing the development and management of the
Project.

6.1.1 The Reference Group
The Through the Looking Glass (TtLG) Project is a community partnership between:
· Lady Gowrie Child Centre, Adelaide;
· Child, Youth & Women’s Health Service (CYWHS);
o Helen Mayo House, (an acute psychiatric unit for women with children 0-
5 years).
o Child Youth Health
These Project partners established the TtLG Reference Group to provide high level
expert advice to guide and inform the overall TtLG Project. Membership is comprised of
representatives from the childcare, health, education and welfare sectors:
· Lady Gowrie Child Centre
· Child, Youth & Women’s Health Service (CYWHS)
o Helen Mayo House (HMH)
o Child Youth Health
· Adelaide University
· University of South Australia
· SA Department of Family and Communities
· SA Department of Education and Children’s Services (DECS)
· Consumer Representative
· Local evaluation team (an external evaluator and internal evaluation assistant)
The Reference Group actively contributed to the development of the TtLG Project, its
activities and resources. The group has been responsible for the
· Recruitment of an experienced Project Manager
· Engagement of 5 suitable childcare sites in which to pilot the TtLG Project
· Ratification of the evaluation strategy and recommendation of standardized
assessment tools.
The Reference Group has met six-weekly for the first twelve months and quarterly
thereafter to consider evaluation feedback and review TtLG activities and resources. A
consumer attended three of the earlier meetings and provided insights regarding the
perspective of clients to proposed project and evaluation procedures.
In response to TtLG staff feedback the membership of the Reference Group expanded to
include representatives of the Childcare centre directors, Project clinicians and cofacilitators.
Representatives were able to raise issues or concerns regarding Project
implementation and relay information back to their respective centers.
An email survey of Reference Group members during the first wave of the TtLG
(October-November 2005) found a high level of satisfaction with their involvement in
the TtLG Project (see: Appendix E7). Key findings were:
· 100% of respondents reported their partnership with the TtLG Project was
valuable for their professional roles and work responsibilities;
· 100% were fully satisfied with their recruitment to the Project;
· 86% were highly satisfied with the progress of the TtLG Project. Additional
comments noted concern that the Project implementation was not synchronized
across all sites;
· 71% were fully satisfied with Reference Group meeting processes. One person
commented on the difficulty of decision making in meetings due to not all
members being able to attend meetings;
· 71% agreed that the wellbeing of clients and Project staff were adequately
considered during meetings. However 3 additional comments highlighted
concerns regarding the number of assessment tools that clients are asked to
complete and the workload for clinicians and other staff7.
Subsequently, on-going engagement with the Reference group was conducted through
regular meetings; the local evaluator and the evaluation assistant (the latter being based
at the Thebarton site) were members of the Reference Group with ‘evaluation’ being a
prioritized standing agenda item to enable regular feedback, discussion of findings and
project actions to be planned.
The Reference Group expertise and advice has informed a range of TtLG strategies
including:
· The recommendation of specific standardized tools utilized in the evaluation;
· Development of OHS strategy regarding home visits to TtLG families by Project
clinicians. In response to concerns raised about clinicians safety communication
strategies between clinicians and childcare centers during home visits were
formalized. Prospective families are now asked to visit the childcare centre to
assess eligibility for the TtLG Project. This is a more efficient use of clinicians’
time and has helped familiarize families with the operations of the Centre;
· Implementation of the modified ‘Strange Situation’ technique in which one
separation and reunion episode between mothers and their children is
videotaped and analysed to address ‘attachment’ – this has been utilized in two
project sites.
7 The evaluation subsequently addressed this issue with clients. Over the five Waves, twelve mothers
expressed difficulties with answering the evaluation questionnaires. However, these tended to focus on
concerns with the extent to which the standardized instruments reflected the gravity of their situation and
feelings, rather than the demands of completing several instruments. The need to talk about these issues
was highlighted thus reinforcing the decision to triangulate the evaluation approach with qualitative
interviews.

6.1.2 Childcare sites
Five childcare sites were selected to implement the TtLG Project.
· Lady Gowrie Child Centre, Thebarton, Adelaide
· il nido Child Care Centre, Paradise, Adelaide
· Salisbury Highway Child Care Centre, Salisbury Downs, Adelaide8
· Lady Gowrie Child Centre, Brisbane
· Lady Gowrie Child Centre, Perth
The TtLG Project based at Lady Gowrie Adelaide has been operating since 2002 and is
relatively well know in childcare, health and education sectors in Adelaide. There are
regular referrals to the Project and the enrollment in each wave is now at full capacity.
The TtLG Project has been implemented relatively smoothly in the other two Adelaide
sites, assisted by the strong inter-sectoral relationships between Gowrie childcare and
health and education sectors, ease of communication and convenient locations. The
childcare staff and management at all three Adelaide centers are supportive of the
principles of the TtLG Project and work in partnership to support families.
Project implementation in the Perth and Brisbane sites has been more problematic. The
Gowrie childcare partnerships with health and education sectors are not as strong here
and this has impacted on Project implementation outside of South Australia.
Acquiring referrals has been the major challenge experienced for the TtLG project in
Queensland. The Queensland government offers a range of well known and popular
parenting Projects such as the ‘Triple P’ and ‘Future Families’. Health and education
agencies support these well established Projects and have been slow to refer clients to the
recently established TtLG Project. However, promotional activity by the Brisbane based
clinician has resulted in increased recruitment as the project has developed.
The Perth TtLG Project is overseen by the Community Services section of the Gowrie
organisation. Due to administration and planning decisions the TtLG Project has been
relocated to a different childcare site during each of the first 3 waves. This has been an
acute challenge for Project staff in particular the clinician who has to build relationships
with different childcare staff and local referral agencies and communities. These
challenges have impacted on the starting times of the different waves.
Whilst Lady Gowrie Child Adelaide has a long established tradition of primary care
giving, across the other 4 sites there is broad variation in staff knowledge and
understanding of the Primary Care giving and Attachment principles which underpin the
TtLG Project. Project management has had to provide additional support and training to
other TtLG centers as they implement primary care giving.

6.1.3 Project Management
A Project manager was recruited June 1st 2005. The manager is a senior staff member of
the project partner CYWHS and has been seconded from her substantive position in the
agency to take on the role.
8 This is a privately run Child Care Centre

There has been a major public health reform within South Australia with consequent
amalgamation of government health departments and restructuring. These reforms have
impacted directly on TtLG partner agencies:
· The Project Manager has needed to renegotiate time release to the TtLG Project
from her substantive role in the agency. Her time commitment to the Project has
reduced from 1.0 (full-time) in Waves 1 and 2, to 0.6 in Waves 3 and 4 and has
further reduced to 0.5 for Waves 5 and 6, the remainder of the Project. This has
resulted in a substantial impact on the manager’s workload;
· The CYWHS Chief Executive Officer who was a co-developer of the initial Project
submission has left the organisation.
Each child center agency across the three States operated autonomously and had their
own policy statements and managerial structure. This generated some difficulties with
regard to accountability and responsibility.
Whilst the project was managed and funded through Gowrie Adelaide at Thebarton, the
clinicians, being located at specific sites were also subject to managerial requests and
structures germane to those sites9. This caused some difficulties which may not have
occurred had the project been run across sites which were accountable to a single
organizational management structure. Establishing MOUs for all participating sites,
stipulating the reporting and implementation requirements of the project and the roles
of participants and supervisors may have helped to alleviate these problems.
These difficulties were circumvented where there was a keen commitment to the TtLG
project at the managerial level and good communications and on-going relations with
Gowrie Adelaide (e.g. with the Brisbane site). This was less evident in the Perth site, and
disagreements arose regarding the implementation of the project, reporting
requirements and adaptations to the model. Staff turnover amongst key players and
management exacerbated this and Perth prematurely left the project on completion of
Wave 5. It is notable that all the key players interviewed from the Perth site were very
positive about the project and regretted its ending. Certainly, the Perth sites are
currently formalizing PCG and seeking to retain other elements of the project in their
practice.

6.1.4 Project Staff
i. Recruitment, Retention and Communication:
For each site, the TtLG team includes the Directors of the Childcare Centers and the
TtLG staff team working directly with families comprising:
· A clinician from a health profession (i.e. social work or psychology),
· A co-facilitator, a qualified childcare worker who assists the clinician in the
weekly group session and also liaises with the primary caregivers.
· Primary caregivers, the childcare workers who are the ‘prime’ carers of the TtLG
children.
There has been a turnover of primary caregivers across all sites. This reflects the
workforce issues in the children’s service sector across Australia10; nationally there is a
9 Clinicians also reported to their clinical supervisors.
10 See: ‘Reflections’: Issue 27, Winter, 2007. Gowrie Australia
high staff turnover and shortage of trained childcare staff. This has implications for the
TtLG Project as new childcare staff require training in primary care giving and
attachment theory.
Whilst staffing at the clinician and co-facilitator levels has been relatively constant in two
of the South Australian centres, there has been some staff turnover experienced at other
sites:
· The Lady Gowrie Child Centre Adelaide at Thebarton, site of the original pilot
TtLG Project, has a stable TtLG team with the original Project staff clinician and
co-facilitator remaining in their roles.
· The il nido, Adelaide site also retained a stable TtLG staff team of clinician and
co-facilitator across Waves 1, 2 and 3.
· Salisbury Centre experienced a turnover of clinician after Wave 2. The new
Clinician started prior to the commencement of Wave 3. The first three Waves
have each had a different Co-Facilitator in place, although the person employed
for Wave 3 is currently engaged with Wave 4 and should complete Waves 5 and 6.
· Brisbane site experienced difficulties recruiting a clinician, with the two initial
appointees resigning before the Project commenced. However the current
clinician has successfully implemented the TtLG Project for 3 waves. The original
co-facilitator has worked on all 3 waves.
· Perth TtLG Project staffing issues have been exacerbated by the relocation of the
TtLG Project across 3 different sites in each wave. There has been a different cofacilitator
and centre director for Waves 1, 2 and 3. The clinician has recently left
the center following Perth’s withdrawal from the Project after Wave 5.
The geographical dispersion of sites has limited the number of collective staff meetings
across the Project. However, this has occurred on a number of occasions presenting
opportunities to provide capacity building training, exchange experiences and provide
evaluation feedback to stakeholders (see: Section 6). Regular teleconferencing and group
e mail discussions have occurred for the Clinicians, Managers and Centre Directors.
The implementation of the first waves of the TtLG project required an intensive training
program, which whilst being well received and beneficial, nonetheless generated
additional workloads for staff engaged with the project. In the early stages the staff were
grappling with the project whilst awaiting training in specific areas. There was some
anecdotal evidence that initial increased workload may have contributed to staff
turnover early in the project. A longer period of induction prior to taking on TtLG clients
would have helped to address this.
The evolution of a PCG culture in the workplace has alleviated staff workload as the
project progressed; the practice is no longer seen as ‘additional’ to existing work, but has
become “the way things are done here”. However some staff whilst highlighting the
rewarding professional and personal benefits have also pointed out the additional
emotional demands the PCG approach generates, the “Ying and Yang of the circle of
Security”.
The amount of training required by the project was comprehensive and intensive and has
developed a more capable, skilled workforce (See: Section 6.2.1). These factors have
raised questions concerning staff remuneration:
‘We’re better trained and provide a better more intensive service than anywhere
else in the sector, so I think we should be rewarded for that in some way’.
Whilst sites could accommodate individual staff changes, where several staff need to be
replaced, a lull in project activity is inevitable whilst new staff are inducted. The
preferred option raised by stakeholders is to take measures to retain project staff.
A potential suggested solution to optimizing staff retention and recruitment is to
establish a form of accreditation for those who have undergone training. A potential
paradox here is that gaining qualifications/credentials from involvement with the project
may broaden employment options elsewhere and hinder staff retention. Linking
accreditation with a specified period of practice experience might alleviate this.
Improving financial remuneration would also help to retain staff. Given the considerable
investment in training, and the additional expense of training new staff, this option
should be given serious consideration should the model be extended or adopted.
ii. The Role of the Clinician
Feedback obtained through a telephone survey, from interviewing the clinician’s
Reference group representative and from the Professional Stakeholder Survey conducted
after the completion of Wave 4, has revealed strong support for the content and
strategies of the TtLG Project and satisfaction with the positive outcomes that are
achieved with the participating families. Clinicians are also very satisfied with the
individual clinical supervision with experts in attachment and early childhood which has
been organized through the TtLG project manager.
The clinicians’ role was central to the delivery and running of the project at each site. In
practice, this extended beyond direct responsibilities relating to the participating
mothers and children. Additional roles identified in this evaluation have included:
i. training and induction of staff in the primary care giving approach and project
processes;
ii. promoting the project and approach (“marketing the project”) in the community;
iii. Supporting the emotional needs (through debriefing sessions) of PCG staff that
have engaged and formed close relationships with project families11.
Whilst clinicians were expected to network with peers and other agencies in helping to
identify potential coordinated options for clients in need (including recruitment and
potential follow-up after the project), broader promotion rested with each participating
Director.
PCG promoted the development of close relationships with mothers, children and
families who were experiencing (sometimes profound and on-going) personal problems;
It is noteworthy that the additional roles ii and iii identified above were not envisioned
as clinician responsibilities in the project model and essentially these responsibilities
resided with the centers CEOs.
subsequently there was a potential to cause a degree of empathetically nourished
emotional distress in PGCs. Whilst the well-being of staff resides with the site manager,
the expertise of the clinicians and their centrality to the project precipitated their
allocation to or adoption of the staff support role. Clarification of this role and the
procedures for its enactment varied across sites and has not been stipulated in the
model.
Taking on these roles required the development of new skills in addition to reorientating
to the PCG philosophy and becoming familiar with the TtLG procedures12.
Moreover, the need to engage with the range of data collection activities for the
evaluation added to workload. This was particularly demanding for clinicians in the early
stages of the project which would have benefited from more preparation time.
Subsequently, the implementation of the project was viewed as being too hasty; staff
were broadly of the view that the first Wave of clients were recruited too early and that
they were not fully equipped to handle the tasks required early on.
The need for more time to embed clinicians in their respective child care sites was also
evident. Many of the clinicians were from welfare backgrounds and did not have prior
experience working collegially with child care workers. Certainly the extent to which PCG
was operationalised was unfamiliar territory for staff operating at some sites. For other
sites PCG had already been established. However, for all sites, more preparation time
prior to the first Wave of clients would have helped to establish the PCG practices and
collegial working environment encouraged by the project.
iii. The role of the Co-facilitator
Evaluation feedback was collected from the TtLG co-facilitators through email survey, a
follow-up interview with the co-facilitators’ Reference Group representative and through
the professional stakeholder survey. Overall the co-facilitators are very satisfied with the
content and strategies of the TtLG Project and their involvement with the families.
In particular co-facilitators valued the training that they had received in Primary Care
giving, Attachment Theory and group facilitation.
Co-facilitators acted as two-way conduits between the clinician and PCGs. Good relations
between clinicians, co-facilitators and primary care givers were viewed as crucial to the
project working at an optimal level. Contextual differences were evident across the sites.
In Queensland, the ‘grass roots’ experience of the clinician was viewed as providing the
advantage of greater understanding of the complexities and pressures experienced by
PCGs. Here, the co-facilitator was also a director at one of the Brisbane sites which was
viewed as having an ‘equalizing’ status effect with the clinician, but also provided more
impetus to disseminating information about the project and encouraging the uptake of
staff training.
The need to clarify roles and responsibilities of co-facilitators and clinicians was evident
early in the project; disagreements here were deleterious to the efficient functioning of
the multi-disciplinary team approach. However, these issues were resolved over time
(and in some cases after staff changes had occurred).
12 The need for clinicians to develop their pedagogical skills for training is discussed in
Section 6.2.5.

6.2 Objective 2: To build capacity of participating Childcare Centres to
develop and adopt a sustainable integrated primary care-giver system
6.2.1 TtLG Training and Staff Development
Project management has developed an inclusive program of training for all TtLG Project
staff including primary care givers of TtLG children, co-facilitators, clinicians and
childcare centre directors and managers. Staffs at the 5 participating childcare sites
receive ongoing attachment theory and primary care giving training. Clinicians also have
access to regular professional clinical supervision. Training in evaluation and conducting
focus groups was also provided by the local evaluator.
An Action Learning approach has been adopted in order to deliver, evaluate and refine
training program. Reflective action learning activities implemented within each site
following the initial first round of ‘site based’ training assisted with the identification of
specific needs and gaps in knowledge, skill and confidence which informed the content of
subsequent sessions as well the development of additional training modules.
Formal Evaluation data has been collected from each of the main internal training
activities conducted for staff across all five sites; this is presented in Table 2. All
workshops below were conducted at Adelaide with the exception of the two waves of
short training sessions which were delivered ‘in situ’ across all five sites. The Project has
been particularly active in delivering formal training to Project Staff from all
participating centres; the amount of formal training activity has exceeded that originally
detailed in the Evaluation Plan.
The training plan for 2007 – 08 financial years was reviewed in consultation with all site
teams and the project manager to ensure that staff development requirements relating to
the project were identified and accommodated into the final year. Each participating site
continued to focus on consolidating and building on the previous years learning.
Each site continued to utilize the skills and expertise of staff (usually the clinician) within
the individual site at the local level. The clinician session outlines for in-service training
were shared via email with follow up communications regarding the information to be
delivered. Sessions focused on Attachment and PCG and were integrated into the
monthly staff team meetings.
Mentoring was providing through staff exchanges between sites and this occurred
between Adelaide and Perth and Il Nido and Salisbury (August-September 2007).
Articles of interest were circulated and several attachment focused reference books were
purchased for staff access. Examples of articles circulated were:
· AFRC Briefing Paper on Building relationships between parents and carers in
early childhood
· The Circle of Security: roadmap to building supportive relationships, Robyn
Dolby

· Pam Cahir, What matters in Early Childhood? A conversation with leading
national and international experts.13
13 References supplied by the Project manager TtLG

Table 2: TtLG Formal Internal Training Activities Across Sites.
Training Date Attendees Training Project Content
Developmental Support Program
The formal Evaluations of training sessions appear in Appendix E.
The action learning approach has embraced the identification of and response to staff
concerns which required further development and support. For example, whilst sessions
1 and 2 were planned as part of the Project model, subsequent sessions 3 and 4 arose
from communication between management and the Project team which identified issues
requiring further staff development.
6.2.2 Summary of Evaluation Findings from each capacity building activity
conducted across sites.
1. Childcare Staff Training
A series of Primary Care giving and Attachment Theory training sessions were offered to
childcare staff at each of the 5 TtLG sites during Wave 1 of the Project. Attendees
reported high levels of satisfaction with their training and perceived the training as
useful and appropriate for their work practices.
93% (n=98) of respondents identified helpful aspects of the training. The Circle of
Security and Circle of Repair concepts were identified as the most helpful aspects of the
training. Printed materials, group discussions and role playing with other childcare
workers clarified and reinforced these concepts. Other helpful aspects of training
included information on professional boundaries ‘looking after ourselves’; reflection and
review of primary care giving information.
88% (n=92) respondents rated the overall training as very good to extremely good.
80% (n=84) respondents found the training content very useful to extremely useful.
90% (n=94) reported they would implement the training into their work practices. The
most frequently described implementation strategies focused on the childcare worker
becoming the secure base in the Circle of Security, using reflective practices and team
work.
‘I will be more understanding, listening to children, 100% available ...not just being
there’;
‘Thinking about the child first and focus on feelings not behaviours’;
‘I will be more understanding, more realising the child’s reaction is from their
unexplained feelings and emotions not just their attitude;’
‘I will reflect more on my own feelings and thoughts, will be bigger, wiser and kinder’
“I will continue to work as a team, communicate, support and reflect on primary care’.
88% (n=92) perceived the training as beneficial for families at their centers
‘Children will feel safe that you’re there for their needs. Parents will feel secure leaving
their children with people that understand them’;
‘Better attachments, better understanding of what parents feel as they drop off their
child’;
‘Building trusting relationships, using knowledge and applying it to parents with secure
base wording about attachment theory’;
‘Forming relationships to give a sense of security and comfort...families will have an
understanding of our involvement with their child and I will have a better
understanding of their child’;
‘Improved transitions in and out of our room…assisting parents understanding of
enrollment’;
‘Increased understanding of children’s behaviours has made me reflect back on some
children in my care and understand their behaviours’.
2. July 2005 TtLG Project Team Training Workshops, Adelaide14
Workshop training items:
· Attachment theory and Circle of Security
· Children’s Wellbeing and Involvement Observation Scales
· TtLG Project processes (referrals, assessments etc)
· TtLG evaluation plan
Attendees reported high levels of satisfaction with the workshop format, Project and
organization:
· 90% agreed to strongly agreed that the training materials were clear and easy to
understand;
· 95% agreed to strongly agreed that they felt equipped with skills to use the
attachment model in their work;
· 75% of attendees reported multiple benefits from attending the workshop
including increased understanding of TtLG Project strategies, networking,
learning about other workers roles and experiences.
3. August 2006 Workshop, Adelaide
Workshop training items
· The TtLG documentation (manual, forms, session activities)
· Introduction to Evaluation and the TtLG Evaluation plan
· Group facilitation
· Video work with parent
· Reflective practice for childcare staff
· TtLG training plan for childcare sites
Attendees reported high levels of satisfaction with the workshop activities, training
materials. In particularly all co-facilitators (n=5) reported an excellent overall rating for
their specific group facilitation training including relevance of the course content, quality
of training handouts and the trainers’ facilitation of the workshop.
Workshop attendees participated in group evaluation activity brainstorming responses
to evaluation questions outlined on a whiteboard. Responses highlighted the Project
success factors and impacts, and areas of concern.
14 Summary Evaluation Reports from each workshop appear in Appendix E
33
4. February 2007 Workshop, Adelaide
Workshop training items:
· Team building – family case study sculpturing exercise
· Reflective practice
· Emotional Availability Assessment
· Interpreting Parent Child Dyads
This workshop Project was informed by an action learning approach based on feedback
that there was a need for role clarification and a clearer understanding of the ways in
which the different workers engage with a TtLG family.
Participants were asked if there were changes in their understanding of the various roles
in the TtLG Project:
· 77% reported some increase to quite an increase in their understanding of other
staff roles;
· 59% reported some increase to quite an increase in understanding their own role;
· 35% indicated they already had a clear understanding of their role and hence
there was no change.
In the overall rating of the training:
· 71% agreed the team building exercise was useful;
· 83% agreed to strongly agreed the Project ‘sculpturing’ exercise was relevant to
their work;
· 83% agreed to strongly agreed that the style of presenting was good;
· 77% agreed to strongly agreed that the training materials were clear and easy to
understand.
Additional comments indicated that clinicians would prefer to explore and discuss
Project components which relate more specifically to their role.
Should be additional training for clinicians as a separate group to explore at a deeper
level in order to support the work we do in the group’.
5. TtLG Project Day 30th April 2007, Adelaide
This Project Day reviewed the TtLG principles of partnership, collaboration and
integration.
Feedback from the February 2007 workshop informed the planning of this Project day.
Representatives of key players within the TtLG Project (centre directors/managers,
clinicians, co-facilitators and primary caregivers (n=20) reviewed the TtLG Project
‘through the lens of the underpinning principles of partnership, collaboration and
integration’.
Group discussions identified a range of factors that require further action by Project
management.
1. Investigate communication options and guidelines regarding working with families
taking into account professional boundaries and confidentiality.
2. The roles of each Project team member to be reviewed and documented to reflect more
accurately the specific roles and associated responsibilities.
3. An induction package to be developed for new staff joining the Project, in particular
new childcare staff.
Project management is currently working on addressing these factors.
Evaluation data from childcare centre directors indicates that childcare staff are
supporting primary care giving practices. Directors reported that childcare staff now:
· place more importance in relationship based care as their knowledge about
attachment theory and in particular the Circle of Security has grown;
· disseminate parenting information more confidently to the wider parent body
(e.g. sleeping information);
· use attachment theory in their Learning Stories to inform parents of the
importance of providing a secure base for children to return to;
· The co-facilitator from one site has commenced a post-graduate Degree in Infant
Mental Health, which is developing her capacity to better support the TtLG
families.
6.2.3 Internal Training Sessions conducted within sites.
There has also been regular ‘site specific’ training delivered on a monthly basis within
each Centre. These have allowed contextual issues for each Centre to be explored with
regard to applying the TtLG Project. The areas addressed in this ‘internal’ training have
included:
· Revisiting attachment concepts and primary care giving as the approach to child
care in the centers;
· Mandatory Reporting;
· Professional Boundaries;
· Reflective Practice;
· Video taping, worker child dyad;
· Children’s Wellbeing and Involvement Scale15 Implementation16;
· Review of TtLG evaluation activities to ensure that staff are confident in their
application;
· A child care worker was supported to visit WA Perth and Brisbane sites from
Adelaide to deliver an update on primary care giving and to provide follow up
support in the rooms;
· Revision and update Attachment and Primary Care giving;
· The COS review and introduction to State of Mind Concept;
· Being Emotionally Available;
· A single induction session was held to support new staff and other staff as an
update. These staff were not familiar with the underpinning theories of the
Project, the Project itself its components and associated activities;
15 Laevers, F. (ed.) 2005, Well-being and Involvement in Care Settings. A Process-oriented Selfevaluation
Instrument (SiCs), Brussels, Kind &Gezin
16 This was implemented following identification of difficulties encountered by some staff in Wave 1 (see:
Section 5.2). This training was complemented by the distribution of a training video (see: Section 6.2.4).
· Training in ‘Learning Stories’. Staff with the relevant expertise visited sites and
presented the information.
6.2.4 Externally Provided Training / Conferences and Workshop
The project has actively engaged external expertise to provide additional specialized
training directly relating to aspects of the TtLG project (see: Tables 3 and 4).
The training received by staff through the project has been extensive. Staff across the
board expressed profound impacts in the ways they interpret and respond to child
behaviour, the adoption of PCG in professional practice, the utilization of new skills in
early childhood education. Several staff indicated that the training had been a revelatory
insight to the human condition, and had informed relations between staff, staff and
clients, staff and management and social and personal relationships outside the
workplace. Managerial practices had also been influenced.
The Kent Hoffman training was specifically highlighted as the most substantial impact
for clinicians and co-facilitators17. The ‘Marte Meo’ training (again utilizing video
methods) was also cited as particularly beneficial. Training of less use was the
‘sculpturing’ exercise and team gatherings which had been, according to some
stakeholders, mislabeled as ‘training’.
A caveat here was that in promoting the PCG approach, there was a danger of devaluing
existing staff skills. However, this pitfall was successfully avoided. The strategy of
promoting and explaining the PCG rather than critiquing existing practices was well
recognized. Having received training in the approach, seen it in action and practiced it
professionally, staff were convinced of its benefits and relished the opportunity to engage
with it. The training has also promoted an awareness of the need for and a desire to
continue with on-going learning in PCG. The experiences have in this sense set several
staff on a new educational pathway:
‘I’ve been studying infant mental health and I’m now doing a Masters… this was
totally influenced by the project’.
6.2.5 Professional benefits and Working Practice Improvements
Workers recognised some need for some of their peers to be persuaded initially to
consider the PCG approach as changing work practices took time and some motivation,
and in some cases they alluded to colleagues who had yet to fully adopt it. However, all
were convinced that once established PCG became irreplaceable, and highly valued by its
adopters. The need to ensure that ‘this is how we do things here’ through policy, training
and the professional practice of all workers underpinned this. Where this became
established, the practice of PCG and working in child care generally was viewed as
became easier:
17 It is notable that several key players applauded the training sessions received in evaluation. Whilst this is
consistent with the findings from self-completion a survey, given the evaluator was conducting the
interviews a degree of ‘Hawthorne effect’ cannot be excluded. However in two cases stakeholders affirmed
that the learning acquired through the evaluation training had been applied in other projects ran from the
center.
‘When something new comes along, you always get some people who are
reluctant to change at first unless they have to. But there’s no doubt in my mind
that once this has happened, and people start to see the benefits it then comes
easier’;
‘Yes, when you start practicing it and seeing how it works, you just want to
learn more about it and experience it more. It’s changed how I work. You just
start thinking differently and reacting differently. Using the circle of Security.
Much better’;
‘I’d say that child care work’s been made easier by PCG. Your working more
with the family and it makes child care a much more positive and growing
experience for everyone’;
‘When you see how the project effected some of the children and mums, you
know its made working life easier because some of these mums we’d be seeing
anyway… The bond you make with the children and the family’;
‘It was a bit daunting at first, and a bit stressful. But no, it’s got easier and
easier. Once you have it (PCG) and it’s established you’d never go back’;
‘It’s made work more pleasant and positive. There’s actually less pressure and
stress than before the project now’.
The cultural change in ways of working has benefited service provision for other children
attending the centers:
‘The project has really equipped us to handle all kinds of difficulties. You get
past the behaviour and start addressing underlining causes I suppose. It’s really
helped us in working with all kids at the centre’.
The professional benefits gained and the benefits for child care practice generally has led
the primary care givers to champion and advocate for the more whole sale adoption of
the PCG approach in the sector:
‘We need to promote primary care giving generally and the project in
particular. I just couldn’t work any other way now. It’s just so much better than
before and has mad the job so much more enjoyable and rewarding. It’s been a
pleasure to come into work!’
‘We need a broad change so that all child care centers adopt the approach’.
The professional impact of the approach on those engaged with the project has been
profound and influenced career paths for PCG staff:
‘It’s been fantastic for me; it’s really changed the work I work and what I want
to do in the future work wise. I want to do more of this. It’s been an absolute joy
to see the real differences you can make in people’s lives’;
‘I’ve decided to try and take things further and to do some post-grad studies in
this area’;
‘I originally thought of the job as a bit of a stop-gap thing really, although in my
case it’s lasted longer than I intended. But this project and the primary care
giving approach and the training and everything, well it’s just blown me away.
I can definitely see a future in working in this area now… yes, I shall look to
develop my career in this area now’.
6.2.6 The Need for on-Going training
Staff turnover across a number of sites emphasized the need for on-going training in
PGC and the procedures of the project (see below). Given the centrality of the clinician
and co-facilitator to the project and their intensive engagement with it, they are well
placed to play a central role in training staff in these areas. Elements of the project
Manual contribute to this. It is also the case that other child-care staff have also become
skilled in these areas and could potentially take on training responsibilities. Additional
the need for more professional staff appraisal procedures to identify training needs was
identified.
Given the profound re-orientation toward PCG needed in some centers, this training
activity is crucial. Both clinicians and co-facilitators have been happy to take on this role
both through formal training and informal mentoring activities. However, currently
neither clinicians nor other staff have received training in practical capacity building
skills, the “how to” procedures of running workshops.
The model would benefit from identifying specific staff as PGC/TtLG trainers, and
ensure they are equipped with the pedagogical skills to deliver capacity building sessions
for other workers as required. These sessions might supplement or replace PCG training
delivered as part of staff induction.
6.2.7 Capacity Built Through Multi-Disciplinary Working
The application of a multidisciplinary approach to child care provided new ways of
working which benefited staff by enabling access to a range of expertise and through
promoting an appreciation and raised awareness of the insights and skills of contributing
stakeholders. Stakeholders felt that the project has subsequently helped to raise the
profile of child care expertise and the professional recognition of child-care staff.
Several clinician and co-facilitator staff have indicated profound influences on their
professional development through engagement with the project. This has been mirrored
in reports of changes in career pathways:
‘It’s put me on a completely different career path’;
‘For me, I’ve discovered a whole new pathway in my career… I want to keep
working with families and kids, not just as a child care worker’.
At the management level, the learning acquired through establishing and managing a
multi-site project involving the complexities of multi-disciplinary team-work was highly
valued.
The engagement of the Reference group was also valued. An unexpected outcome from
this was the embedding of two research students at Gowrie in South Australia engaging
with related projects:
· ‘Secure and insecure attachment relationships in a preschool, long day care
setting’. Masters thesis, School of Psychology, University of Adelaide, 2006
· ‘The attachment relationships between toddlers and their caregivers in child
care’. Sophie Mumford , Honours thesis, School of Psychology, University of
Adelaide, 2007
The project has also promoted staff collaboration across the Lady Gowrie sites for the
first time. Clinicians and managers from outside of South Australia have been keen to
point to the support and training supplied by the Gowrie Adelaide Centre. This centre
has also acted as an example of a working model for others and staff benefited from
visiting the centre and seeing the project operating first hand. However the extent of
collaborative relationships varied across sites; Perth questioned the need for inter-site
collaboration given its differing mission and community development focus.
6.2.8 Operational Issues
Directors of childcare centers have raised concerns with Project management regarding
the practical capacity of sites to embrace the workload generated through the TtLG
project:
o The process of keeping childcare places available for TtLG families can
sometimes impact financially on the centre’s operation, particularly when a
family withdraws from the Project at late notice.
o The challenge of finding spaces for the wide age range of children in each TtLG
wave. This was particularly difficult when there were numbers of babies in the
Project due to tight staffing ratios in the babies’ rooms. Directors have identified
the need to consult closely with the clinicians in regard to accommodating the
children of TtLG families. Clinicians may need to vary the enrollment of families
across different waves depending on the age range of children.
o Many families continued with childcare after they complete the group TtLG
Project. This can sometimes lead to a ‘cumulative’ impact on staff workload as
these families (some with on-going problems) continue to look to childcare staff
for support with their child at the same time as ‘new’ TtLG families join the
centre.
o It is sometimes difficult for directors to release staff for TtLG activities due to the
shortage of childcare staff. At times there is no staff member available to backfill
a vacancy. An overall industry sector shortage of staff impacts on directors’
capacity to release childcare workers from the centre rooms.
Many mothers retained the child care services after leaving the project and were
subsequently still in regular contact with their PCG. The project has promoted the
development of greater understanding between PCG and client informed by ‘inside’
knowledge of family circumstances; in some cases personal circumstances have been
exchanged in a reciprocal process of trust development and the forging of friendships.
Whilst this was viewed as highly positive, the nurturing of close relationships during the
project created the potential for further working demands for staff from clients who had
completed it. There was evidence of some need for further guidance or an ‘exit strategy’
which clarified the professional aspects of the nature of the relationship post project for
all agents.

Summary
Clearly the TtLG project has been very active in providing a range of capacity building
activities to staff across the five project sites. This has built capacity to adopt and deliver
a sustainable integrated primary care giving system, which in turn supports the TtLG
families and improves attachment outcomes. This has allowed the organisation to
deliver better services for targeted families and their children, (a national ‘Invest to
Grow’ priority).

Table 3: Conferences / Workshops Project Staff Supported To Attend
Date Event Details

6.3 Objective 3:
3.1 To equip and empower a range of parents of young children with the
knowledge, awareness, confidence and skills to successfully overcome the
barriers to attachment
3.2 To foster and nurture positive parent well-being outcomes
3.3 To foster and nurture positive child well-being outcomes.
The following findings are taken from data amalgamated across all project sites for
Waves 1-5 (August 2005 – April 2007).
Primary evaluation data was collected from mothers using a self-completion post-project
questionnaire administered universally on completion of the Project. If mums who did
not complete the program are excluded, this yielded a response rate of 100% (n=106).
This was further triangulated with the application of the pre and post standardized tools
to assess project impacts. A further telephone survey of mothers from the Project’s first
three waves was conducted three months after Project completion yielding a response
rate of 82% (n=50)18. A further follow-up survey of mothers from Wave 2 and 3 was
conducted sixteen-eighteen months after project completion in order to address
sustained outcomes for families. This yielded a response rate of 73% (n=29)19.
As mothers were the primary group targeted and engaged in the intervention, most of
the findings presented in this section relate to this group and these are triangulated with
findings from other stakeholders where appropriate. Fathers attending formal group
sessions organised through the Gowrie Adelaide center (including fathers from the il
nido center) where also surveyed, yielding a response rate of 58% (n=14)20.
6.3.1 Process Evaluation - Mothers
i. Recruitment and Retention of Families to the TtLG Project
A total of 118 families have been recruited to the Through the Looking Glass Project to
date. In 2 of these families the grandmothers of the children participated in the Project
as they were solely responsible for the children.
Formal TtLG sessions for fathers has occurred at the Gowrie Adelaide centre at
Thebarton where 24 fathers (partners of recruited mothers) have attended. A summary
evaluation report of these sessions appears in Appendix E2. There is some anecdotal
evidence of individual consultations with fathers occurring at other sites and an informal
group session at Perth.
18 See: Summary Report Appendix E1.
19 See: Summary Report Appendix E14.
20 See: Summary Report Appendix E2.

Table 4 outlines family enrollment across the 5 sites for each Wave.
Table 4: TtLG Families enrolled in TTLG Waves 1-5

One hundred and six families (90%) completed the TtLG Project including 136 children
(see: Table 5). Four recruited families failed to commence, and eight withdrew
participation from the Project prior to completion.
Table 5: Total Enrolment and Completion Rates (Mothers and Children)

For the twelve families who began the project but did not complete, a range of reasons
for their departure were given, however none of these were attributed to the project
itself. Reasons provided were: Work commitments, Child sickness, Family moved
interstate, and child enrolled in another centre.
ii. Demographic Characteristics of Participating Families
Demographic information about mothers and children was collected using items from
the National Evaluation Service Users Questionnaires. A spread of ages was engaged by
the Project; whilst the majority of mothers were under the age of 35 (56%, n=60), 39%
(n=41) were in the 35-44 age bracket, with ten mothers being 18-24 (see: Table 6).

Table 6: Mothers Age range

The large majority of mothers identified themselves as ‘Australian’ (84%, n=89) but only
one as an ‘Aboriginal or Torres Strait Islander’. The remaining respondents who
indicated their backgrounds (n=16) were from eleven different mainly European
countries. Fourteen respondents indicated speaking a language other than English at
home: five indicated ‘Greek’, and two citing French, Italian and Serbian. Other languages
indicated by single respondents in each case were: Italian, Romanian, Serbian. Japanese,
Romanian and Spanish were cited by individual mothers.
41% (n=43) of mothers indicated they were married with a further 16% (n=17) being in
‘de facto’ relationships. 27% (n=29) were single, 9% (n=9) separated and two mothers
indicated they were divorced.
Half of the mothers recruited to the Project have educational qualifications beyond Year
12 (n=53), with 22% (n=23) having a University degree and 28% (n=30) a Vocational
Certificate or Diploma from a TAFE or college. However one in five indicated they had
not reached Year 12 (n=22), and there was a further 15% who indicated some other
qualification or did not respond, (see: Table 7).
Table 7: Mothers Education level
44% of respondents (n=47) were in some kind of paid employment during the time they
were engaged with the Project; 56% (n=59) indicated the main source of household
income came from wages or salaries, (see: Table 8). However, for one third of families
the household’s main source of income came from Government Benefit, Pension or
allowance (33%, n=35). These tended to be single mothers; 85% (n=22) of single
mothers indicated government benefits, pension or allowance as the main source of
household income; single mums made up 67% of those on benefit/allowance/pension. If
‘separated’ and ‘divorced’ are included this proportion rises to 82% (n=27).
Table 8: Employment Status and Main Household Income
One hundred and thirty six children enrolled and completed the TtLG Project (72 female
and 64 male). Ages were skewed toward younger children with one quarter being under
the age of 1 year (25%, n=34) and more than half being under two years old. Table 9
presents a breakdown of children by age.
Table 9: Age Ranges of Children engaged with the Project
Seventy one percent (n=97) of children continued on in childcare after their mother had
completed the Project. Clinicians reported several reasons why some families did not
continue with childcare including:
· Location and transport factors, family homes were not convenient to the centre;
· The ending of subsidized child care rendered continuation too expensive;
· Mother was home on maternity leave and wanted child at home;
· Child started kindergarten or school.
Retaining children from the TtLG project in childcare has resource implications and
presents challenges to staff at participating sites (See: Section 6.2.7). This issue has been
identified by management at each site and has ultimately been governed by established
capacity guidelines for each centre.
iii. Mothers experiences with TtLG Project
On completion of the project, respondents indicated that they felt positive, appreciative
and safe in the Project setting (see: fig i):
Fig i: Likert Scale findings concerning Centre Staff and Facilities:
· 87.7% (n=93) agreed that they felt relaxed and safe at the centre (with over half,
54.7% strongly agreeing);
· 86.8% (n=92) agreed that they felt comfortable with the Project workers (with
63.2%, n=67 strongly agreeing);
· 86.8% (n=92) thought that the childcare arrangements were satisfactory (66%,
n=70 feeling strongly);
· 85.8% % (n=91) agreed that it was easy to get along with their child’s primary
caregiver (57%, n=60 indicating ‘strongly agree’).

The three month follow-up survey confirmed that most mothers (88%, n=44) continued
to view the childcare centre as providing a ‘safe space’ for their families. They recalled
feeling relaxed and being able to freely talk about their issues.
‘(Children) were clingy and I was anxious about them, but I could watch them play
through a window and this made me feel better’;
‘Everybody was welcoming not like other places I’ve had to go to with our issues’;
‘Good place... childcare workers are friendly I could ask his carer any questions’;
‘Meeting in the childcare centre was relaxing for me I looked forward to Tuesdays was a
great experience’.
The Post Project questionnaire for clients included a balance of positive and negative
Likert items concerning Project delivery. The large majority of views were very favorable
about the Project: (see: fig ii):
Fig ii: Likert Scale findings concerning Client Assessment of Project
Processes

· 86.8% (n=92) respondents agreed that the weekly group sessions were enjoyable
(59.4%, n=63 strongly agreed);
· 88.7% (n=94) agreed that the ‘information materials were clear and easy to
understand’ (42.5%, n=45 strongly agreed);
· 80.2 (n=85) disagreed that it was ‘difficult to find transport to and from the
childcare centre’ (46.2%, n=49 strongly disagreed);
· 77.4% (n=82) disagreed that ‘there were not enough opportunities to discuss my
experiences of being a parent’ (38.7%, n=41 strongly disagreed).
· 73.6% (n=78) disagreed that ‘the timing of the sessions was not convenient for
me’ (40.6%, n=43 strongly disagreed);
These favorable views were sustained over time21. The follow-up survey revealed that
98% (n=49) 0f mothers were clearly satisfied with their experiences of the TtLG Project
three months after completing it, with 72% (n=36) indicating they were highly satisfied
with the way in which the Project helped them feel closer to their child. Project staff and
the opportunity to meet other mothers were highlighted:
‘All the people were wonderful there; the whole thing was about getting in touch with
little brains’;
‘Very satisfied…it made me look at childrearing in a different light’;
‘Satisfied ...it made me feel happier meeting other people like me’’;
‘Really satisfied with it, all activities worked, (clinician) made you feel comfortable’;
‘Wonderful ...it should be compulsory for all mothers leaving hospital; they shouldn’t be
without this information’.
21 See: Appendix E1 for a summary report of the findings from the 3 month follow-up survey and Appendix
E12 for a summary report of the fifteen month follow-up survey.
iv Mothers assessment of Project Activities
In order to address the issue of what works best and for whom, mothers were asked how
helpful each aspects of the TtLG Project had been to helping them to understand their
child’s attachment needs. The findings are summarized in Fig iii.
Fig iii: Mothers’ assessment of helpfulness of Project elements to
understanding their child’s attachment needs.
The following figures refer to the percentage of mothers indicating the strategies helped
them to understand their child’s attachment needs a lot:
· 80.2% (n=85) indicated ‘Reflecting on the videotape of your interaction with your
child’;
· 77.4% (n=82) indicted the ‘Explanation about the Circle of Security model’;
· 72.6% (n=77) indicated ‘Group reflection on individual family videos’;
· 69.8% (n=74) indicated ‘Talking with other mothers in the group about
parenting’;
· 69.8% (n=74) indicated ‘Explanation about the Circle of Repair model’;
· 64.2% (n=68) indicated ‘Discussion about the Shark Music video’;
· 64.2% (n=68) indicated ‘Individual sessions with the clinician’;
· 54.7% (n=58) indicated ‘Talking about the video ‘You are so Beautiful’ (with
20.8% indicating helped ‘a bit’);
· 37.7% (n=40) indicated ‘Your child’s Learning Stories’ (with 25.5% indicating
helped ‘a bit’);
· 31.1% (n=33) indicated ‘Talking about the book ’I Love my Mummy’ (with 31.1%
indicating helped ‘a bit’).
More than eight in ten mothers indicated that 80% of the strategies employed had
helped them (with six in ten indicating that 70% of strategies had helped them ‘a lot’)
with regard to understanding their child’s attachment needs. Over half of the mothers on
the project found 80% of the strategies had helped them ‘a lot’. Even the two least
successful strategies from the mothers’ perspective helped mothers to some degree in
over six out of ten cases. There was no relationship between the likelihood of finding a
strategy very useful and demographic variables (age, educational level or ethnic
background).
Mothers’ additional comments highlight the ways in which these Project strategies
tended to be viewed holistically and taken together increased parents’ understanding of
their children’s attachment needs.
‘The whole concept. I have changed my views on child needs and parenting’;
‘Although there will always be situations and issues with my child which will challenge
me, I feel the information discussed has provided me with a working model with which I
can face these, now and in the future. It is one which I am comfortable with and which
solves much confusion’;
‘The circle of security makes sense and it’s good to watch the video of me and my kids
and see the circles actually happening in action. Develops understanding’;
‘Reinforcement of attachment model through many different examples, situations etc,
especially video of each attendee was good as it helps me with thinking of how to
respond to different situations at home. Opportunity to really discuss parenting issues
with other mums away from children in a non-judgmental group helped’;
‘Learning about Circle of Security and Repair. Looking at my child’s feelings – what’s
going on behind the behaviour and helping him to work through his feelings’;
‘The personal videotaping was very helpful; it helped be to gain insight about my
behaviour and my children’s responses and vice versa’;
‘The Shark Music clarified my worries. I am more aware of my fears and my child’s
needs and emotional transference’;
‘Shark music before and during the group I had a lot of shark music and I was able to
understand why and what it just meant. Now I feel I am able to prevent that shark
music by being a lot more aware of it’;
‘Personally for me the most beneficial aspect of the Project was the shark music.
Recognising my own irregulation of emotion (or shark music) and staying with that
during times of high emotional support for my child has made those difficult situations
somewhat more bearable with a clearer understanding of my shark music’;
‘Learning Stories – don’t think they have helped me understand the attachment needs
but I think that this and the family photos are very valuable in building attachment &
relating to my child about the day at childcare. Also for the primary care giver to learn
about home life and my child’s interests etc’.
The three month follow-up survey revealed that mothers spontaneously asserted that
they continued to value both the childcare and primary care giver components of the
TtLG Project:
‘Childcare – the best thing for me has been taking the opportunity to have ‘time out’
without the children. Breathing space – time to find who I am again – I am actually a
human being!’;
‘The childcare has been amazing giving me the opportunity to have some time out. I
never realised how important that is. It has allowed me to get a job and find myself
again’;
‘The Primary Care Giver system should be compulsory as it is better for the security of
the child. Wow. Works well’.
The 16-18 month follow-up of mothers found they reaffirmed the usefulness of the broad
range of project elements; over half of the sample spontaneously indicated several or all
aspects of the project were the ‘most helpful’. The role of the clinician, use of video in
group and individual sessions, the circle of security, and meeting and talking to other
mums were all individually cited:
‘(Clinician’s ) advice and looking at the video of all the other families. That made it
easier to understand how children move around the circle’;
‘It was all helpful. The circle information helped understand that children need to
explore you don’t have to control everything for them... Watching the videos helped me
understand the different ways that children ask for help’;
‘The circle, showing how children move around and need to explore and how you have
to be there for them. Watching the videos really helped understand plus it gave you
some ideas about what the other mothers were doing with their child’;
‘The other mothers talking about how they did things. Their ideas really helped, gave
you some tips to remember’;
‘It was all a lot of help, (clinician) really helped with the video showing me how (child1)
was behaving and doing things with (child 2). Now I can better anticipate what they
will be doing’.
Given the multifaceted and holistic approach adopted in the project model, it is difficult
to identify the most important factors which facilitated improved outcomes. However,
the combinations of group and individual work with clinicians and reflections on the
child/parent video films guided by insights from the ‘circle of security’ have clearly
contributed to greater understanding of attachment.
6.3.2 Impact and Outcome Evaluation
i. Improved well-being and parenting outcomes for Mothers
The post-project survey revealed that respondents clearly felt that the Project had
improved aspects of their parenting and attachment (see: fig iv):
· 85.9% (n=91) indicated that the TtLG Project helped them to feel closer to their
child, with 51.9% (n=55) strongly agreeing this was the case;
· 78.3% % (n=83) indicated that the Project helped them to feel good about
themselves as parents (37.7%, n=40 indicating ‘strongly’);
· 69.8% (n=74) felt more confident looking for other services and supports for
their family since being on the Project (28.3%, n=30 indicating ‘strongly’).
Fig iv: Likert Scale findings concerning Improved Parenting
Few respondents provided negative responses regarding these aspects with three of the
five parents who indicated they disagreed that they felt more confident looking for
services also disagreeing that they felt good about themselves as parents:
· 91.6% (n=97) learnt more about parenting and attachment, with 77.4% (n=82)
indicating ‘yes, definitely’;
· 89.7% (n=95) are more confident in responding to their child’s needs, with 64.2%
(n=68) indicating ‘yes, definitely’;
· 87.8% (n=93) cope better as a parent since taking part in the TtLG, with 53.8%
(n=57) indicating ‘yes, definitely’;
· 90.6% (n=96) have acquired understanding of children’s attachment needs, with
83% ( n=88) indicating ‘a lot more understanding’;
· 90.6 % (n=96) have acquired understanding of children’s exploration needs, with
80.2% (n= 85) indicating ‘a lot more understanding’.
The evidence supports that the TtLG has achieved the objective of empowering parents
of young children with the knowledge, awareness, confidence and skills to successfully
overcome the barriers to attachment.
‘I feel the models provided should be taught to all parents in the community to help them
better understand their children and child development and am therefore very grateful to
have received the opportunity to participate in this Project’;
‘Feel much closer to (son) than could have ever thought. Enabled me to understand (son), his
behaviour, actions and why I react the way I do’;
‘I have learnt so much about the way my daughter reacts and why and how to deal with it.
In conjunction all this is due to (clinician) and she is truly an asset to this Project and for
that I am forever grateful’;
‘The Project has helped me to understand my child’s point of view more and to look at
situations, conflicts and challenges from many angles’;
‘I look at the strategies I have learnt from the looking glass and I am able to meet a lot of my
child’s needs than what I could before’;
‘Looking at the world through my child’s perspective and learning how to do that was
invaluable to me. Even though I am very much struggling with my son still, this course has
given me a lot of tools to work with and keep utilizing. I know it’s not a quick fix but I’m
definitely not giving up on my son or our family’.
These impacts have been sustained since TtLG mothers left the project. The three-month
follow-up survey found that most mothers (88%, n=44) reported a lasting positive
change in themselves since taking part in the TtLG Project. Mothers described
themselves as happier, less stressed and more able to cope.
‘I’m happier now… I’m a single parent and really needed a break… the childcare helped
me get some timeout’;
‘I was exhausted at the beginning… at the end I felt on top of everything... getting the
feedback from others helped (clinician, co-facilitator and primary caregiver;
‘I feel so much better ….I now understand that I’m not the problem… the way (child)
behaves is not a result of me... I can now look at it from his side... see what he wants’;
‘I can cope now …I still get stressed out but I know how to back off’.
Mothers also reported that the project has sustained impacts on their parenting
practices:
· 88%, (n=44) described positive changes in the way they do things with their
children;
· 88%, (n=44) reported increased responsiveness and ability to read cues;
· 80%, (n=40) reported increased confidence in responding to their children’s
attachment needs;
· 74%, (n=37) reported getting less frustrated with their child.
All of the mothers contacted in the 16-18 month follow-up survey (100%, n=29)
indicated they continued to use information or ideas about attachment acquired from the
TtLG project. Mothers reported sustained confidence and competence in parenting,
ongoing empathetic understanding and sustained better parenting practices (despite
children now being considerably older presenting new challenges).
‘She’s a toddler now and wants to play I can see that she does things of the circle I can
help her when things aren’t right. Like when she’s tired she can get frustrated with toys
and things I know to cuddle her and settle her down’;
‘I did get more confident as she got older; it’s easier when they can tell you what they
want’;
‘made me stop and think more …like thinking about why (child) got upset’;
‘I knew I could join in with him, I guess that’s being more confident’;
‘We’re both confident with (child). It was good to be able to talk about the things I learnt
from (clinician)’;
‘Can be a challenge sometimes but I understand that he needs to learn things’;
‘Yes it helped me to find out about how children need you in different ways’;
‘I feel better about (child’s) behaviour; it made me stop getting angry with (child)’;
‘I’m more confident because there aren’t as many arguments with the (children)’;
‘Yes it’s just so different it’s made a big difference to how I feel as a mother’;
‘I am confident with (child) it’s different now he is very active but I can support him’;
‘I think I can understand (child) better and so there isn’t as much stress’;
‘I am feeling better with (child) now’;
‘I know I am doing better now, it’s important for (child) to develop in her own way’;
‘I think that they are better because I can anticipate what they will be wanting to do, I
know that (child 1) needs more quiet times away from her brother’;
‘I guess now I just can stop their fights starting ...like I can see when one of them is
getting upset and I can sort of get in and fix it up first so they don’t really get going’;
‘I think she is happy because we feel happy about it all and we’re OK about just being
with her’.
ii. Positive Child Well-being Outcomes
Mothers also reported improved positive child behaviour on completing the project with
76.4% (n=81) perceiving that their child’s behaviour had improved. Again this impact
appears to be lasting and probably reinforced by more positive parenting practices. The
three-month follow-up survey confirmed sustained changes in improved child
behaviour, with 88% (n=44) of responding mothers reporting lasting positive changes in
their children’s behaviour since taking part in the TtLG Project.
‘Major changes… he is coming out of himself…looks to new people in our life…..he is
happier’;
‘He is more confident. I let him explore and follow his lead. I don’t try to always make a
game for him I follow him and no longer say don’t do this’;
‘(Child) used to be clingy now she’s happy and goes to kindy 4 days a week she’s turned
into a real social creature and wants to go more days’.
Given that the children of mothers surveyed in the 16-18 month follow-up interviews
were significantly older than they were during the intervention (with many moving from
being babies to toddlers), mothers indicated that they found it difficult to attribute their
child’s long term behaviour change to the project. However, many mothers indicated
sustained improvement in family functioning; many felt they could do more with their
children, enjoyed parenting, were better able to cope and felt they were better parents as
a result of the project. Many reported that their children were happier as a result:
‘I think we are both happier and more confident’;
‘I think it’s more that I understand him better. I can join in with him better’;
‘He is changing all the time doing more things for himself …it’s me that’s changed I am
less stressed about doing the right with him’;
‘I think there isn’t as much stress with us’;
‘Going to the course made me see that (child) was really being just a normal toddler, it’s
more that I have changed’;
‘I think I’m better at organising things, like remember to think about things from their
point of view , understanding that sometimes they are just tired and winging and not
really playing up’;
‘I can see more about why (child) is doing things and I think that he is more confident
about doing things for himself’;
‘I think it made us both good parents’;
‘I’m different with (child) better than before and I know that I am a better mother
when I look after myself’;
‘I think I am better at things with (child) and that makes it better at home’;
‘Yes it’s just something that I feel good about I know that I can give (Child) what he
needs’;
‘Really it was by changing how I looked at things with the (children), now I try to think
about lots of things from their perspective e.g. like packing up games at night, I try to
remind them 10 minutes before they have to instead of just coming in and saying ‘do it’;
‘I am a better parent I enjoy them more’;
‘I feel happier with myself for learning about how to be a better mother.’
All of the staff interviewed universally reaffirmed the findings acquired from mothers,
that the impacts of the project have been profound for children, parents and families.
This has had a very positive effect on the staff who participated:
‘You look at the child before and after the project and you just can’t believe it’s the same
child’;
‘Absolutely fantastic to see the way the children develop and change. I can honestly say
I’ve never seen such a dramatic improvement in the toddlers. It’s just a wonderful
project’.
‘It’s been amazing and totally rewarding. A fantastic experience to see the progress of
the mums and children’.
‘One little boy just didn’t speak at all. And his mum was clearly having great problems
relating to him and meeting his needs. And now it’s completely different, chatting away
and his mum’s like a different person. It’s been wonderful’.
‘There’ve been dramatic changes in parents and children. Amazing changes really’.
‘There’s been a huge dramatic change for mums involved – much better understanding
and lots of improvement in attaching with their children’.
The focus groups of primary care givers testified strongly (and in some cases emotively)
to the improvements brought about amongst clients and children by the intervention:
‘There was a child with profound behaviour problems… kicking, swearing, biting… his
mum wouldn’t even talk to him… its completely different now, his mum had acquired the
skills to talk more… his behaviour is completely different. I mean it’s like he’s a
completely different little boy. It’s just wonderful’.
‘This little girl didn’t say a word she used to just scream with these high pitched
squeals… her language improved and she can actually communicate now and her mum
communicates with her’.
‘I’ve seen massive change. Massive changes. There’s been children who just wouldn’t let
go of their mums at first now interacting and playing with other kids. Mother’s being
much more in control of themselves. Massive changes. Even in the appearance of some
mums, their physical appearance, being happier, dressing smarter’.
iii. Overcoming Barriers to Attachment
Staff emphasized that the child parent relationships have been enhanced through project
participation; the project has built on existing strengths and helped parents to
successfully address the root causes of attachment and parenting struggles:
‘It’s produced much stronger and secure relationships between parents and children and
provided a really strong base for the future. Phenomenal success!’
‘Exploring the strengths families have and unearthing the problems and strategies to use
these to address the causes of difficulties…it’s been incredibly rewarding’.
‘When Aw started childcare she was very distressed about leaving mum. Recently she left us
to go to preschool and we say her on her first day and she was very excited about going to a
new place’;
‘When WC commenced care he would never venture far from his primary caregiver and
was distressed when other staff entered the room. He now enjoys spending time exploring
the room and loves to have the opportunity to interact with the older children’;
‘EM has become more creative and her imagination has expanded vastly. She now finds it
easier to engage others in her dramatic play by verbalising her needs more confidently’;
‘AV now is able to more confidently return to her safe base, rather than always staying out
exploring’.
Mothers have shared their learning with others. This was more common whilst TtLG
mothers were still engaged in the project, with over eight in ten mothers (n=97)
indicating in the post survey that they have shared attachment information with friends
and family. More than half of the surveyed mothers in the follow-up survey, (54%, n=27)
reported that they had talked about the TtLG Project with other family members and
friends in their community. This strongly suggests that project messages are being
promulgated by clients some months after they have completed the project:
‘Sharing attachment knowledge with others some trouble explaining it to others & family
but by me doing what I learnt they are picking up and learning’.
Overall evaluation data indicates that the TtLG Project is fostering and nurturing
positive parent and child well being outcomes. For many respondents the experience of
the Project has been very positive and valuable:
‘I feel I have gained an enormous amount of information and much greater understanding
of my child and child’s needs. The experience was enjoyable and something I looked forward
to going to each week’;
‘Thank you for the opportunity to participate. It has made a big difference to our lives.
Thank you also for providing access to childcare. I didn’t realize before how desperately I
needed a break from the kids so I could function when I was with them’.
iv. Standardized Instruments: Pre and Post Scores of Cognitive and
Behavioural dimensions Measured
The findings obtained from the semi structured interviews and surveys have been
reaffirmed through triangulation with the application of the range of standardized
instruments selected for the evaluation.
Table 10 presents a summary of the pre post comparisons for each standardized
instrument applied for the evaluation. The Instruments selected were:
· Hospital Anxiety and Depression Scales (HADS)
· Parenting Stress Index/Short Form (PSI/SF)
· Emotional Availability (EA) attachment assessment
· Children’s Wellbeing and Involvement Observations measures
Table 10: Summary of Statistics from pre-post comparison of scores
obtained from the application of standardized instruments using the
Wilcoxon Signed Ranks Test with Effect Size calculations.
** Two clients who left the project prematurely completed the HADS test.
Using the Wilcoxon Signed Ranks Test, significant improvements were detected across
all but two dimensions.
The hypothesis that the Project would reduce levels of anxiety and depression was
supported in both cases (p<0.01). Using Cohen’s prescriptions for interpreting effect
size22, in both anxiety and depression the effect was approaching a ‘large one’ (denoted
as d>0.8). Over the duration of the project, the number of mothers experiencing
‘moderate’ to ‘severe’ anxiety and depression (scoring between 11-21 on the HADS), more
than halved; from 52 to 25 (anxiety) and from 37 to 13 (depression). Conversely the
numbers acquiring a ‘normal’ score more than doubled for anxiety (from 21 in the pre
22 Cohen, J. (1988). Statistical power analysis for the behavioural sciences (2nd Ed) Hillsdale, NJ: Erlbaum.
measure to 44 in the post) and was 60% higher for depression (from 42 to 67
respectively).
Similarly, ‘stress in the parent child system’ as measured by the PSI recorded median
score changed from 107 to 84. Again the alternative hypothesis that the Project would
reduce stress levels is supported (p<0.01) with a large effect size recorded.
Child’s wellbeing and involvement observation ratings also yielded positive findings: the
median score for wellbeing rose from 3 to 4 and for involvement from 3 to 3.9. The
change in scores was significant at the 0.01 level with large effect sizes recorded in both
cases.
The Emotional Availability attachment assessments based on the scores recorded by
professional independent assessors yielded significant improvements (p<0.01) across
four of the six domains assessed: ‘Child Responsiveness to Parent’, ‘Child Involvement
with Parent’, ‘Parent Sensitivity’ and ‘Parent Structuring’ all improved with effect sizes
being moderate to large. Interestingly, two domains measured through the assessment,
‘Parent Non-Intrusiveness’ and ‘Parent Non- Hostility’ did not demonstrate significant
improvement and had an effect size less than ‘small’ using Cohen’s criteria; However,
these areas were not specifically addressed in the TtLG Project.
v. Impact of TtLG on Fathers
The project has been less successful in formally engaging fathers in organised group
sessions. Reasons for this include the high proportion of single and separated mothers
recruited to the project and work and time commitments of fathers. The PCG approach
has encouraged father engagement with individual child care workers but further
engagement has in most sites been restricted to ad hoc information giving exercises. The
except here is the Gowrie Adelaide centre at Thebarton where a more detailed program
has been run (See: Appendix E2).
A total of twenty four fathers have attended one or more sessions run from the
Thebarton Center (including seven fathers of mothers engaged with the il nido center).
Evaluation forms were obtained from fourteen fathers. Whilst there may be a degree of
self-selection bias in this small sample, fathers surveyed clearly gained greater
understanding of their child’s attachment needs:
· All responding fathers (n=14) reported an understanding of the Circle of Security
attachment model with 78.6% (n=11) reporting a lot of understanding;
· All respondents indicated that participation in the fathers session had given them
an understanding of children’s attachment needs, with 57.1% (n=8) having a lot
of understanding.
Fathers identified project benefits to their families in terms of it helping them to feel
closer to their children and positively influencing their children’s behaviour changes.
Moreover, these sessions were identified impacting on fathers’ parenting skills:
· 92.8% (n=13) agreed that their family’s participation in TtLG had helped their
child’s behaviour (35.7%, n=5 strongly agreed)
· 78.6% (n=11) agreed that they felt closer to their child as a result of their family’s
participation in TtLG (35.7% n=5 strongly agreed)
· 78.6% (n=11) agreed that participation in the fathers program improved their
parenting skills (14.3%, n=2 strongly agreed)
Whilst these findings are tentative, there is a clear indication that exploring additional
ways of formally engaging fathers in group sessions could add value to the benefits
already gained for mothers and their children.
vi. Summary
Clearly the project has improved parent competence and style and improved family
functioning. Mothers have increased their knowledge competence and awareness to
overcome barriers to attachment, are less stressed, depressed and anxious and better
able to cope as parents. Many report better parenting practices, better engagement with
their children and improved child behaviours which they attribute to the project. For
many parents these impacts have been sustained since leaving the project. These
findings provide clear evidence that the project has addressed the national ’Invest to
Grow’ priority areas of: ‘Improved family functioning’, ‘Improved parent competence
and style’ and ‘Improved child social and emotional development’.
6.4 Objective 4:
To develop and enhance social support /friendship networks for the target
group
Fig v presents findings from the post questionnaire regarding support received by
mothers engaged with the TtLG project:
Fig v: Likert Scale findings concerning social supports received
· 86.8% (n=92) indicated that staff respected and supported their family, 73.6%
(n=78) indicating ‘yes, definitely’;
· 77.4% (n=82) indicated that they had developed supportive friendships with
other TtLG project mothers, with 43.4% (n=46) indicating ‘yes, definitely’;
· 65% (n=69) indicated they had received help to find other services for their child
or their family from the project, with 29.2% (n=31) indicating ‘yes, definitely’;
· 26.4% (n=28) indicated they had developed supportive friendships with other
mothers attending the center who were not part of the TtLG Project, with 50%
(n=53) indicating they had not.
Most mothers have received help to find services through their engagement with the
project, and this was confirmed by project staff. However, this often occurred through
informal contacts, networks and channels as the project has not specified the
establishment of formal referral pathways as an objective. Clients who may have more
long standing acute problems might benefit from this being included (See: Section 6.5.3).
Forty one percent (n=12) of mothers in the 16-18 month follow-up survey retained the
confidence to access other services; however, 59% (n=17) indicated that they felt no need
to access services. This may reflect improvements recorded in their well-being, parenting
and family life.
Clearly, the majority of mothers had developed friendships with peers whilst engaged
with the project although this was not as evident with regard to friendships with mothers
who were not on the project. However this still occurred in some degree for over a
quarter of respondents.
Staff confirmed that many mothers had formed lasting social support and friendship
networks through engagement with the project. These appear to be more successful, but
are not exclusive to, where parents have retained connection with the child care centre
and its services. Factors which militate against sustained friendship networks were
usually logistical; where mothers lived far away from each-other, started work or moved
house, the friendships were not as lasting.
TtLG families were encouraged to participate with their families in social and community
events offered at their childcare centre including:
· Christmas Party at the end of the year;
· Easter Party;
· Family tea evening meetings other parents;
· Teddy bears picnic;
· New Parents Morning Tea;
· Sessions for the fathers of those participants who had an active dad.
Supportive friendships endured for over half of the TtLG mothers surveyed in the threemonth
follow-up (54%, n=27). These friendships made during the Project were most
frequently with other mothers who had children the same age. Examples of on-going
friendships included meeting for coffee, attending children’s birthday parties and
maintaining phone contact. Good group dynamics was seen to support the development
of friendships rather than any specific TtLG activity.
Some mothers (26%, n=13) also reported on-going participation in their local
community since the project, taking up activities such as: joining a playgroup (n=7),
commencing part-time work (n=7) and returning to study (n=5). Establishing netball
teams, client organized group meetings and shopping outings were also cited:
‘I enjoyed the social contact and sharing experiences & seeing what others do and
knowing we all share the similar highs and lows of children’;
‘Friendships from group members priceless I feel that this group was life changing,
helping me when I was at my most lowest point. Congratulations for such a wonderful
Project’;
‘The contact with other mothers from similar backgrounds was extremely beneficial in
not feeling alone and their feedback was invaluable’.
The 16-18 month follow-up survey revealed that 28% (n=8) had kept in contact with
others met through the project. The mothers who did not maintain contact with other
group participants from the TtLG Projects cited reasons such as not living in close
proximity to other families or their own work commitments. A number of mothers stated
that whilst they were no longer in contact with others from the project, this did not
reflect negatively on the relationships formed at the time of their engagement.
Clearly social support / friendship networks have been established through the project
which for a sizable minority has endured over an extended period. This demonstrates
that the project has been addressing the national Invest to Grow priority of supporting
child friendly communities.
6.5 Objective 5:
To develop and promote the uptake of a ‘best practice’ model for services
working with mothers and fathers and children around issues of
attachment
6.5.1 Professional Stakeholder Assessment of the Efficacy of the model
A survey of professionally engaged stakeholders across all five sites was conducted on
completion of Wave 4 (a summary report of this appears in Appendix E12). The
interviews with CEOs (or their delegated manager), clinicians and co-facilitators were
semi-structured to include summative scales in order to gain a quantitative assessment
of the impact of the project from their perspective. However, the interviews were largely
qualitative in nature to enable and encourage an open exploration and critique if the
project model. This work was supplemented by two focus groups of PCGs from all project
sites (with the exception of Perth) the findings from which appear in a Appendix E13.
Eighteen stakeholders were interviewed (either face-to-face or over the telephone) with
interviews lasting between 30 – 90 minutes.23
Thematic analysis was conducted on the qualitative findings to identify areas which were
generic to the project across two or more sites. The analysis was conducted in tandem
with the fieldwork and was iterative; as themes emerged these where subsequently
addressed in upcoming interviews using procedures established from Grounded Theory
approaches. This analysis has informed the findings pertaining to model refinement
considerations below.
23 . One clinician from Perth was not interviewed as she had left the project; she was
replaced by a manager who had worked with the co-facilitator.
Stakeholders were very satisfied with the overall outcomes achieved by the Project with
72% (n=13) indicating ‘fully satisfied’ and 28% (n=5) indicating ‘mostly satisfied’.
Satisfaction was expressed largely in terms of the impacts achieved for Project clients.
55% (n=10) thought that the overall goal of the Project had been ‘fully achieved’, and
28% (n=5) ‘mostly achieved’. Two indicated ‘partially achieved’ and one did not know.
For those who did not indicate ‘fully’ the remaining need for the model to be adopted and
on-going was highlighted.
Fig vi: Project Staff Assessment of Extent to which Stated Objectives have
been achieved
Based on 18 respondents
Fig vi presents the summation of the Child Care Centre project staff (broadly defined as
the site managers/CEOs, project managers, clinicians and co-facilitators) assessment of
the extent to which each of the TtLG project’s objectives have been achieved. Nearly all
staff who had engaged with the project to some degree indicated that the project had
achieved all of its stated objectives to some extent. The most successfully achieved
objectives were Objectives 2 and 3 where all but one respondent indicated the objectives
had been ‘fully’ or ‘mostly’ achieved. For the Child centre staff the impacts for clients
and their children and the extent to which the project has built a sustainable PCG
capacity in participating child care centers have been substantial:
‘It’s been absolutely fantastic for the families who’ve been involved’;
‘You see mum’s who took no interest in themselves or their appearance, who can’t
communicate to their children and are deeply depressed totally transform. It’s just
amazing’;
‘I’ve seen children completely change… a truly remarkable experience’;
‘Children who were clearly having real communication problems, one kept biting… one
didn’t hardly speak at all… they’ve become like completely new kids!’
To a lesser degree, the project has developed and enhanced parent support networks, but
this was still viewed as being fully or mostly achieved by 72% (n=13) respondents. This
has taken several forms including retaining contact with the center, its staff and/or
activities, retaining friendships acquired with other project mothers and in some cases
engaging with local established groups. Where this did not occur a number of reasons
were postulated (see: Section 6.4). Some respondents indicated they were only partially
aware of the sustainability of networks and so answered ‘partially achieved’ for this
objective.
‘A lot of the friendship network stuff really depends on the mothers who come along in a
particular Wave – I mean some live miles from each other so the chance of them
carrying on their friendships are pretty slim given the demands of kids. Others work, or
start work etc etc. So this has varied a lot between waves’.
78% (n=14) felt that the higher order objective 5 ‘to develop and promote the uptake of a
‘best practice’ model…’ had been ‘fully’ or ‘mostly’ achieved (56%, n=10 indicating
‘fully’). This is a notable finding given that (in the view of those staff engaged with the
project) the least achieved objective was the ‘lower order’ Objective 1. Whilst 44% (n=8)
respondents indicated this had been ‘fully’ or ‘mostly’ achieved, 56% (n=10) indicated it
had only been ‘partially’ achieved. This was explained in terms of partnerships not being
fully established across project sites (see below); within each site sustainable integrated
partnerships were viewed as having been established through the project. There was also
comments made about the lack of ownership and partnership from other sectors.
‘It’s been great in terms of our own centre and the partnerships formed between the
child care workers and the clinician. And we’ve worked well with Gowrie Thebarton
around training. But we’ve really not had much to do with the other centers’.
‘There’s not really been the ownership across sectors that I would have liked to see. This
has made it much more difficult to get people to take up and run with the project’.
6.5.2 Model Sustainability
The potential of the model to meet client needs in a sustained way is supported here and
the benefits for families with attachment issues of ‘rolling out’ the model would be
substantial.
i. The Adoption of Primary Care Giving (PCG) Child Care Practice
Staff were broadly enthusiastic about the changes in professional practice and
subsequent improvements in the quality of care precipitated by the implementation of
the PCG approach. Staff felt better equipped with the skills and knowledge to practice
child care in a more effective, insightful, reflexive and ultimately more rewarding way.
The changes were profound for many staff across the centers, extending to working
practices with clients and children, relations between staff and between staff and
management, managerial practices, and for some influencing personal social
relationships. Practice has become more holistic, orientated toward ‘emotional needs’
and relationship focused. This has enabled staff to interpret child behaviour differently
and engage more intensively with families accessing the centre.
There has been a ‘cultural shift’ in working practices precipitated by the project, away
from behaviorist models such as the ‘Positive Parenting Practice’ approach toward the
wholesale adoption of PCG24. The approaches were almost universally viewed as
benefiting children, families, parents and staff. These changes in skills, learning,
philosophical orientation and professional practice are strong legacies from the
implementation of the project. However, for some staff, concerns were also expressed
about the extent to which PCG was fully understood and implemented; the need for
regular review, an on-going training and support in reflective practice was subsequently
asserted.
ii. Systemic Changes at the Policy Level
The project has precipitated systemic changes amongst participating Centers. This has
varied in degree as each has separate policy development procedures. However, in all
cases attachment theory and PCG is being embraced at the policy level.
The implementation of these approaches in professional practice through the TtLG
project has preceded and prompted the broader policy changes. PCG is now part of
induction and ‘refresher’ programs for new staff across several participating sites.
iii. Expanding the Project
The project is extending to other Lady Gowrie sites. A presentation of the TtLG Project
and the evaluation findings took place in Caboolture, Queensland in February 2008, and
Caboolture plans to adopt the project later in the year. The project is also conducting
consultations with Aboriginal communities to identify how the project might encourage
greater participation and meet the needs of Indigenous families.
Whilst several sites had adopted aspects of PCG prior to the project the extent of this varied
greatly; it was universally asserted that PCG implementation had been substantially enhanced
and improved through the Project.
iv. Continued use of Project Resources
The ‘Circle of Security’ poster has been enthusiastically adopted as a symbolic and
practical guide for staff and families using the centers. Many of the written resources
(including books and articles concerning attachment theory) have been compiled within
each site and are utilized as needed. Other resources developed through the project have
been taken up including the development of a DVD ‘The Father/Child Journey’
specifically for fathers of families accessing the services.
The project Manual was generally well received amongst those staff members who had
seen it. The manual was viewed as essential for the initiation of key players in the project
(clinician, co-facilitator and managers) and was referred to often in the early waves of
the project and by new staff. All aspects of the manual were viewed as useful but
clinicians tended to be selective, referring to the manual occasionally as a ‘refresher’ once
they had become familiar with the materials.
Co-facilitators outside of South Australia seldom used the manual being guided more by
the formal training and materials received. However, those located in one of the three
locations within South Australia tended to access materials from the manual more
regularly. The manual was viewed as a supporting resource and not a replacement for
practical training.
v. Impacts on Clients
Impacts on mothers and children have been found to be sustained over time (See:
Section 6.3).
vi. Capacity Building Benefits for Staff
Staff have clearly been up-skilled through the project. Many have reported changes in
career pathways and seeking further formal training in related child care areas (See:
Section 6.2).
Sustaining the model through Lady Gowrie would be impossible without the funding
needed to support the employment of the project manager and clinicians at each site.
Moreover, removing the ‘gap’ fee for child care covered by the funding is likely to have a
deleterious effect on the recruitment of families to the project, particularly given the
proportions of clients who are single mothers and/or are receiving Government benefit
support. The need for an on-going Government sponsored implementation of the model
possibly through State Government agencies was strongly championed by management.
6.5.3 Model Promulgation
Several formal presentations promoting aspects of the TtLG model have been delivered
by TtLG Project staff on specific Project components / activities. These have been
summarised in Table 11. In addition, the Gowrie Adelaide CEO, Project Manager and
local evaluator presented findings from the TtLG evaluation on three occasions to South
Australian Government Departments in early 2008.
The diagram appeared in several rooms in the four sites visited by the evaluators.
A list of project resources appears in Appendix C
Roll out of the model would benefit from a greater period of preparation at the hosting
agency. The Project Manager reported that a longer planning and training period prior to
taking in the first wave of TtLG participants would have enhanced project
implementation. This would have enabled individual sites and staff members to identify
what extra training would be required to develop the capacity of the site and staff to
support TtLG families. An action plan for each year could then have been developed
earlier.
Table 11: Conference and Workshop activities Promulgating Learnings from
the TtLG Project
6.5.4 Promoting the Primary Care Giving Philosophy
The Adelaide Gowrie site currently engages in training activities across the child care
sector. Given the amount of training and capacity built in the area of PCG and
attachment through the project and the benefits of adopting these child care approaches,
extending the reach of these training activities was broadly supported. This might
include further staff ‘visits’ to Adelaide to observe, shadow or be mentored in the
practices of PCG. These opportunities were available during the project and were clearly
valued by staff from other states.
The need to link some ‘post project’ families experiencing acute or enduring issues with
supporting agencies raised some questions from staff concerning continuity of care and
the extent to which the referred to agency’s philosophy and practice mirrors that of the
referred agency. Promoting the PCG philosophy and raising awareness of the approach
across appropriate sectors and agencies was advocated as a means to help address this.
There is a large potential for the trained project staff to provide training services in a
range of areas (e.g. PCG, attachment, Circle of Security, group work) to other agencies.
The example of co-facilitators being able to deliver group facilitation training was cited
as potential inter-sectoral training activity. The delivery of training would also promote
stronger linkages and partnerships. There is evidence of this happening with the
Brisbane site currently engaging with Queensland Health’s ‘Seeds’ project, working with
them for the adoption of the Circle of Security. Dissemination of the approaches used has
also been enacted by Gowrie Adelaide, through presentations of the model and
evaluation findings at TAFE colleges and South Australian health and education
government departments.
Expanding this external training role was also viewed as helping to raise the profile of
Lady Gowrie and present potential opportunities to generate funding to help retain the
clinician role when the TtLG project finishes. The need to explore ways in which trained
TtLG staff might further apply their skills (and optimize the considerable investment
made in skills development) when the project ends was also championed.
The need to promote and build capacity in PCG across the child care sector was strongly
advocated by those engaged with the project. The potential to link training in PCG to
formal courses run through the TAFE and University sector was also highlighted and
championed.

7 DOCUMENTATION OF THE PROJECT MODEL
The TtLG model is detailed on the Lady Gowrie web site at:
http://www.throughthelookingglass.org.au/cms/about
The model is presented in Section 1 of this report with the project Manual contents and
project resources provided in Appendix C. The project Manual may be requested form
Lady Gowrie Adelaide.
7.1 Model Development
The Project Manager in collaboration with clinicians and members of the Reference
Group has developed a TtLG project manual which also details the project model.
Clinicians reported that each group of families had a different dynamic with individual
families having diverse needs. This complexity and diversity required clinicians to be
flexibly responsive and employ the use of a range of activities and resources during the
weekly group sessions. Information and resources successfully used with families were
identified and systematically adopted and included in the manual. The manual has
therefore evolved over the duration of the project and has become a reservoir of
resources which complement its set of guidelines for the implementation of the TtLG
Project. The contents of the Manual and resources compiled have been provided by the
Lady Gowrie Project Manager and are summarized in Appendix C.
There have been few changes in the original model since its conception in the project
proposal. An exception was the inclusion of the modified ‘strange situation’ technique in
Gowrie Adelaide, reflecting the interest and expertise of their particular clinician. This
was implemented following Reference Group discussions as a means of further exploring
child parent attachment, and was used to complement the range of established model
techniques. However, management has expressed some concerns that the technique may
sway what is a multifaceted model away from the social/community elements and more
toward therapeutic aspects; it has not been integrated into the model at other sites.
Given the contextual differences between sites drawing conclusions about optimizing the
‘best practice’ model is problematic; a number of issues have been identified which were
specific to the context of individual sites. Other issues relate to the nature of the TtLG
Project being conducted nationally across sovereign bodies with their own managerial
and accountability structures (See: Section 6.1.3). This impeded the establishment of a
coherent set of working, reporting and accountability procedures across the five engaged
sites. These difficulties have also been exacerbated by staff turnover and geographical
distance, notably with the Perth site which ended its involvement with the project after
the fifth wave. However, all sites (including Perth) have expressed strong wishes to
continue with the project in some form.

7.2 Adjustments made to the Model
Whilst the essential elements of the model have been retained throughout the project,
the evaluation has revealed a number of difficulties encountered in its implementation
across the five sites. These have revolved around the more generic difficulties of
establishing an efficient functioning ‘multi-disciplinary’ team which have been
confounded by the need to do this across ‘independently managed’ bodies:
· Complexities of reporting and accountability27;
· Ambiguities concerning professional boundaries between clinicians, child care
workers and co-facilitators;
· Blending the project with the objectives and priorities of individual child centre
sites;
· Promoting the adoption of a different paradigm of collaborative working.
There were also initial managerial concerns about the possibility of clinicians feeling
‘isolated’ given that their role distinguishes them from other child centre employees.
Measures were subsequently taken to link clinicians to colleagues located in other sites.
Paradoxically, the emergence of the TtLG clinicians as a mutually supportive group
across the project sites, whilst strengthening collegiality and facilitating sharing of
learning and information which has contributed to resource development, appeared to
have emphasized the boundaries between their professional roles and those of other
TtLG service providers. This created difficulties to implementing aspects of the model
and inhibited the project logic; the efforts required to blend existing norms and
preferred practices which have been reified in this group with the requirements and
application of the TtLG model were underestimated. Two examples have emerged:
difficulties in persuading clinicians to reduce home visits; difficulties in persuading
clinicians to assess video film of client child/parent interaction collegially with the
primary care givers.
A series of recommendations were identified and addressed at the management level:
· Greater staff engagement with the Reference Group (through staff
representation) to allow more open dialogue;
· Establishing a program of national meetings of TtLG staff which had a ‘dialogue’
focus across staff groups and sites;
· Establish an on-going training program in attachment and its mechanisms to
monitor its application in working practice;
· The introduction of staff appraisals for clinicians;
· On-going revision of the Manual to clarify job specifications, roles, the applied
nature of primary care giving, reporting procedures and the TtLG vision;
· Formalising procedures for raising staff concerns;
· Supporting open and effective two way communication between directors and
clinicians in order to achieve optimal implementation of the TtLG Projects in
centers;
· Reviewing the co-facilitator's role and responsibilities and investigating
additional ways that the co-facilitator can work with the TtLG family;
· Clarifying the childcare centre director’s role and responsibilities in supporting
centre staff working with TtLG families. In particular the TtLG family’s
relationship with the primary caregiver and any consequent demands on the
primary caregiver.
Clinicians are currently accountable to three bodies: the TtLG Project Manager, the
Child Centre Director and their professional supervisor
The above areas have been addressed, and have been documented in the project Manual.
Service delivery ‘Action Plans’ for each individual site for Years 2006-07 and 2007-08
were also developed; these appear in Appendix B.
As clinicians became more embedded in their centers the barriers identified gradually
eroded; this was assisted by staff changes in which some clinicians were recruited who
were known to have experience with child care work, or who were more willing to
embrace the new paradigm.

7.3 Areas for Potential Model Refinement Identified by Professional
Stakeholders
Following wave 4, professional stakeholders identified a number of areas where the
model and project might be refined or adapted. Several of these are discussed earlier in
this report under the specific objectives to which they relate. Additional areas identified
by professional stakeholders are presented below. These should be generally be viewed
as considerations for those seeking to implement the model rather than stipulations as
there will inevitably be contextual and staff differences in different site locations.
A full itemization of all areas for potential refinement is presented in Section 7. 4
i. Optimising Multi-Disciplinary Teams
The need to ‘balance’ the contributions of the varied expertise brought to the project
through the multi-disciplinary team was a challenge for the project. The unique service
provided through the TtLG project was embodied in the fusing of therapeutic (clinician)
and early intervention (child care) approaches; these were conceived as traditionally
having separate allegiances and identities. Ensuring an integrated approach in the model
was made more difficult by the organizational and managerial differences across sites.
Bringing together all staff earlier in the project and more frequently to address issues
and share learning experiences would have encouraged a more coordinated ‘team’
approach. Instigating more professional development activity at the team level earlier
would also have promoted the progression of a working team culture within sites.
Having participated in the project since its instigation, some of the PCGs consulted in the
evaluation had worked with more than one clinician and co-facilitator. These workers
provided a particular insight to factors which helped promote the optimization of the
multi-disciplinary team within the child care setting. The discussion was steered toward
aspects which might inform the best practice model and several suggestions were
highlighted. There was evidence presented of these measures being successfully
exercised in different sites:
· Where more than one locality was used by a given center, the need for the
clinician to be available across these localities;
· The benefits of a clinician having some background in child care provision
including the day-to-day difficulties encountered by child care workers;
· The need for clinicians to hold the personal qualities of being: non-judgmental,
respectful of other’s expertise, and empathetic;
· Incorporating periods of time when clinicians can interact with child worker
staff;
· Incorporating time when ‘new’ clinicians can work with staff in the child care
rooms and observe their working with children in their care;
· Adopting the use of a ‘Communication Folder’ in the event of the clinician being
unavailable, to enable staff to record issues of concern to be addressed later;
· Clarifying times when the clinician would be available for consultation with the
PCGs.
Conducting ‘open nights’ at the community centre in which the clinician could speak to
all parents about the project was also valued as this was viewed as helping to address any
pre-existing sense of stigma.
ii. Adjusting for client demands on PCG time
Client demands on primary care givers’ time was identified as an issue but one which
could be accommodated. The potential for PCGs to be removed from attending to
children was successfully addressed by identifying a second member of staff to act as a
‘secondary care giver’ in their absence. For sites where this was applied it worked well.
Time demands from parents were also alleviated by forward planning of meeting times;
parents and PCGs agreed convenient set times early in the project when their PCG would
be available for meetings. This procedure should therefore be incorporated into the best
practice model.
‘Finding the time for the parents was sometimes hard for me especially if I had
some children to be looking after’;
‘(Agreeing available time) Worked really well for us… It wasn’t carved in stone
but it meant that everyone was clear about when the PCG was available...’
iii. Suitability of the Project for ‘Acute’ Cases
A small minority of families were experiencing acute problems at a level of severity the
project could not fully address. Whilst this raises questions concerning stricter
definitions of eligibility for recruitment in order to filter out clients who may require
more intensive therapeutic intervention, there were some disagreements amongst
professional stakeholder groups regarding the exclusion of these clients. Clinicians and
co-facilitators felt that excluding more acute cases would deny them the considerable
benefits to be gained from the project. PCGs asserted that substantial and rewarding
benefits were achieved for these families. For these stakeholders, it was felt that
identifying a willingness to try to engage with the project was a more important factor
than severity of condition. However, two managers expressed concerns regarding
disruptive difficulties experienced with specific families. Four potential strategies
emerged around this issue:
1. ‘screening’ mothers to ensure a willingness to engage with the project, be
reflective and seek underlying solutions to attachment issues;
2. ‘linking in’ specialized concomitant support with other agencies for specific cases
if required;
3. extending the engagement period for families who need it;
4. establishing a more formalized ‘referral pathway’ for families who may need
further help;
Strategy 1 presents challenges which may only be possible to address individually
through the professionally informed impressions of the clinician. However, given the
holistic family centered and personalized approach adopted by the model, the flexibility
to embrace strategies 1-3 on a case by case basis was viewed as feasible; these measures
could potentially be accommodated in the current model. With regard to strategy 4, in
several sites, referring specific clients to new services occurred where linkages to external
agencies were already established. As the model stands, whilst the project seeks to
empower clients to seek appropriate external support services as part of objective 3,
there is currently no formal strategy to develop referral pathways to appropriate services
for those clients who may require further therapeutic help. Whilst there was evidence of
this happening on a less formal basis, incorporating this formally would help to ensure
that ‘post project’ cases identified as requiring it, receive that additional support. There
may be a case for extending project linkages and partnerships with suitable ‘follow-up’
agencies to enable this to happen. This may also yield benefits in terms of external
agencies directing additional suitable ‘recruits’ to the project.
The benefit of locating the project at Centers for Early Development and Learning was
highlighted as these will embrace a range of easily accessible services at the same venue
and potentially optimize multidisciplinary service delivery.
iv. Engaging Aboriginal families and fathers
The project has not recruited ATSI families. At the time of writing this report, an
extensive consultation with Aboriginal communities from urban and rural areas is being
conducted using TtLG project funding. It is hoped that this will lead to modifying TtLG
to produce a culturally appropriate model which will encourage uptake from Indigenous
families.
The engagement of fathers has varied across the different sites. Given the high number of
single mums and the work/time demands for families with fathers, this has been
problematic. Where this has occurred it has been largely through information giving
sessions and informal liaison with the PCG. This has been beneficial in helping to
establish relationships with families. Formal group activity with fathers has yielded
positive impacts (See: Appendix E2). Means of extending this activity to engage more
fathers should be further explored.

7.4 Overview of further Potential Model Refinement Areas and Actions
Potential refinements to the model have been unearthed more recently as the staff who
engaged with it from the various sites have become more familiar and experienced. A
comprehensive list of these is listed below. These areas have been presented to the
management team for consideration of further action or inclusion in the project Manual.
The decisions taken regarding each item is included below. A number of these areas are
being addressed in a newly funded Lady Gowrie Adelaide project which will provide
primary care giving training for up to five new child centre sites in South Australia
(2008-2009).
i. Position descriptions roles and partnerships:

The model would benefit from further establishing clearer position descriptions
of the working roles of staff engaged with the project (Clinicians, Co-
Facilitators, Managers / Supervisors), taking account of the myriad roles which
have been adopted during the TtLG project.

The model would benefit from stipulating the nature and proceedings for the
provision of PCG staff support (including emotional ‘debriefing’) and the
Clinician role in this.
· The above areas are being included in a revised manual.
ii. Implementation of the model:

The model would benefit from a longer period of staff induction and site
preparation prior to recruiting clients. This will allow PCG practices to be
established as a firm base for the project and encourage clinicians to be more
embedded in the child care center. Given the learning acquired through this
project, this preparation period should be no less than two months.
· Induction for sites participating in the new training project
will be conducted by experienced lady Gowrie Adelaide
staff over a longer time period.

The model would benefit from establishing protocols for communication
between PCGs and clinicians which may include time-tabling meetings and/or
adopting the use of a ‘communication folder’.

PCGs and clients would benefit from negotiating agreed times for consultations
early in the project to avoid parental demands impeding PGC time with
children; establishing a secondary care giver for support has also been
identified as a best practice in this regard.
· The above areas are being included in a revised model with
the exception of establishing a secondary care giver; child
centers should accommodate PGC time from children.

The model may benefit from establishing formal linkages with service agencies
in order to link them to clients with acute problems.
· The role of the clinician is being reviewed re networking
responsibilities

Consideration be given to extending the project for the small number of families
who need it; the flexibility to extend the project for these families would need to
be incorporated in the model.
· To be considered. However there are on-going resource
implications and concerns about fostering dependency
iii. Implementation of the model specifically across several sites / agencies:

Where the model is applied across sites, more regular meetings of all staff to
share and explore experiences of the team approach would contribute to the
more effective functioning of the multi-disciplinary approach.

The model would benefit from the development of MOUs for all participating
sites which clarify ownership, accountability procedures, roles and
responsibilities of management and staff (including position statements for
clinicians, co-facilitators, PCGs). Establishing agreed procedures for managing
conflict/disagreement could usefully be included.
· The above areas are being included in a revised manual.
iv. Establishing procedures for clients leaving the project:

The model would benefit from developing in plain English a client ‘exit strategy’
which includes clarifying the role of the PCG for parents no longer engaged with
the TtLG project.
· The above areas are being included in a revised manual.

The model would benefit from developing more formal linkages and pathways
to suitable external agencies to address specific identified client need where
appropriate.
· The importance of establishing networks and a working
knowledge of local agencies will be stressed in the manual.
The decision to formalize linkages for specific clients resides
with individual site directors.
v. Training / Staffing issues:

The model would benefit by including multi-disciplinary team training as early
as possible within sites to enhance functionality; this should include time/
measures to familiarize clinicians with child care workers and their
professional practice through observation and interaction.

The model would benefit from formally identifying specific staff as PGC/TtLG
trainers, and ensure they are equipped with the pedagogical skills to deliver
capacity building sessions for other workers as required. These sessions might
supplement or replace PCG training currently delivered as part of staff
induction.

The model would benefit from including the requirement of extending regular
professional staff appraisal to identify staff training needs.
· The above areas are being included in a revised manual.

Incorporating measures to retain trained staff (e.g. accreditation and financial
remuneration) into the model would enhance efficacy and continuity of service
delivery
· No action - This was beyond the control of the model
vii. Future Directions:

Consideration should be given to further expanding the training role of Gowrie
centers across the sector in order to raise awareness of and build capacity in
PGC. The promotion of PCG in formal training provided through TAFE and
Universities should be explored further.

The model would benefit from continuing to engage in dialogue with Aboriginal
communities to inform its cultural appropriateness for Indigenous families.

The model should continue to provide information sessions to fathers and
encourage exploration of flexible ways to greater engage with fathers where
possible.
· The above areas are currently being embraced and enacted
through the newly developed training project

Avenues to utilize the new skills acquired by clinicians and co-facilitators
through engaging them in cross-sector capacity building activity should
80
continue to be explored; this would potentially promote further beneficial postproject
outcomes.
· The decision to engage staff in cross-sector capacity building
resides with individual site directors.

8 DISCUSSION OF EVALUATION METHDOLOGY
This evaluation report addresses findings from Waves 1-5 of the TtLG Project for the
period July 2005 to April 2008. The project is currently completing a sixth Wave the
findings from which will be included in an evaluation summary report to be submitted to
Lady Gowrie Adelaide.
The participatory action research approach has allowed on-going feedback throughout
the project; it has clearly informed the identification of project implementation issues
and inhibiters to the project logic. This has encouraged action with regard to training
and discussion, and unearthed a range of process issues which were addressed as the
project progressed. The approach has also facilitated the application of a broad range of
triangulated methods to gather data from the range of stakeholders engaged with the
project over time. Some of these methods have only been possible to implement with the
collaboration of project staff as evaluation partners.
The procedures employed in the evaluation have been more elaborate than was initially
envisaged and have required considerable planning and organisation to enact (notably
the collection of video footage and its professional assessment pre and post each
implemented Wave). This has been largely brought about by the absence of any one
psychometric instrument to measure parent/child attachment and related dimensions.
This has also necessitated training in evaluation and the application of evaluation tools
utilised here. The speedy implementation of the project prior to the procedures for data
collection being formalised led to some missing data in Wave 1. However the application
of the tools recommended by the Reference Group has allowed the spectrum of impacts
to be specifically measured, and the procedures established have been ethically endorsed
and were acceptable to project clients. Data collection improved as the project
progressed and is of an acceptable standard. Standardised data has been complemented
by qualitative approaches which have allowed causality to be addressed, the experiences
of staff and clients to be explored, pertinent model aspects to be unearthed and
sustainable impacts to be identified.
Following the first three waves, the evaluation has continued to collect, analyse and
present findings from clients. However, in seeking to address the higher order objectives,
the evaluation conducted semi-structured (but largely qualitative) interviews with all
staff across all sites who had engaged with the project (CEOs, managers, clinicians and
co-facilitators) and conducted focus groups with Primary Care Givers following Wave 4.
During conclusions about the model from the broad range of contextually specific
accounts generated in this process was problematic. However, a number of areas have
subsequently been identified regarding optimisation of the implementation of the model
which have been discussed with project management and will be further explored with
the project Reference Group. These refinements have been included in this report.
The evaluation addressed project impacts over time for clients largely through a threemonth
follow-up interview of all mothers from waves 1-3. The findings obtained were
highly positive and provided strong evidence of sustained impacts. This raised questions
concerning longer term outcomes for clients and the local evaluator proposed
implementing a longer term follow-up to address this. The selection of clients and the
period of time for follow-up were largely governed by the parameters of the funding
period and the evaluation resources available. With the endorsement of management
and the Reference Group, evaluation resources were shifted from the three-month to a
fifteen month follow-up (which, given the logistics of tracing some clients became an
eighteen month follow-up in some cases). All clients from Waves 2 and 3 were surveyed
and interviewed for this. The findings from this work are included in this report.

8.1 Consideration of a Control Group
The PAR design of this evaluation was informed by the need apply a methodology which
embraced and informed project evolution and development over time. The evaluation
rejected establishing a control group for a number of practical, ethical and design
reasons. The potential need for modification to the project model in the light of
evaluation findings would reformat the intervention reducing any comparisons with a
control to snapshots of the model at that particular developmental phase. Moreover, the
model was multifaceted and applied across different site contexts with a degree of
flexibility in each case.
Ethical difficulties were recognised by the local evaluator in that the control would deny
the intervention to parents identified as being in need. Logistical barriers were also
identified including: identification and recruitment of enough subjects for the control
group, problems of applying the standardised measure in an appropriate way with
control group recruits (particularly the videoing of parent interactions with their
children in their homes on ‘pre’ and ‘post’ occasions, and the infeasibility of setting up
scenarios for the ‘wellness and involvement’ scales to be applied), and the increase in
costs (including training, travel and incentives) of doing so with enough numbers of
geographically spread parents for robust comparisons to be made.
However on concluding Wave 3 it became evident that place restrictions had generated
lists of eligible parents at two specific sites who could not be accommodated into the
project. Given that the model had matured by this time, and that its nature had not
modified greatly from the initial application, the external evaluator revisited the question
of establishing a control group using these eligible parents which he raised with the
Reference Group. The advantages of using a control group primarily rest on
strengthening the case for causal attribution.
The local evaluator subsequently calculated effect sizes generated by the application of
the pre and post standardised tools to calculate the size of the control group required for
robust comparisons to be made. Since the control group is expected not to change over
time, the effect size for the pre-post difference in the project group was used as an
estimate of the difference between the project group and the control group. Sample sizes
to achieve at least an 80% power were calculated for each of dimensions addressed by
the standardised tools. A selection of these is presented in Table 12.
Table 12: Control Group Sample Size Calculations for Measures registering
There are however persisting issues with establishing a control group for this evaluation:
1. Incentives and raising expectations amongst control group recruits. The lists of
eligible people identified are not part of an official ‘waiting list’ and may not be
taken onto the project if other more ‘needy’ cases are identified following their
participation. Given the sensitivity of issues to be addressed suitable incentives
would have to be identified. However, there was an ethical need to emphasis the
possibility of being denied access to the project; this might also alleviate
Hawthorne Effects on control subjects ‘post’ responses.
2. Possible selection bias. If project subjects were systematically selected according
to perceived ‘need’ this creates differences based on need between the control and
project groups.
3. Costs of accessing eligible subjects in their homes, and additional costs of
training and data analysis.
4. The need to restrict recruitment to those two areas that have identified
contactable and eligible subjects; given the contextual differences between sites
conclusions would be restricted to specific site(s).
5. Objections from clients concerning home videoing. Preliminary work conducted
early in the project uncovered a strong reluctance amongst clients who were no
loner part of the Project to be video taped at home with their children. This may
also amplify a latent selection bias in the control group.
6. Related to 4, the demands of the Well-being observations notably the need to
establish familiarity between children and researcher raises issues of feasibility.
7. The need for ethical approval given the change in evaluation design
Given the sample sizes required and the particular difficulties that video taping presents,
use of the Emotional Availability scale is highly problematical. The Well-being
observations were not feasible given the need to pre-establish relationships with control
group children (and their parents).
However, the possibility of establishing a ‘Comparison Group’ in SA, and applying pre
and post measures of the PSI and HADS was considered. Such an endeavor would have
required establishing a clear, sensitive and ethically accep2protocol which clarified
subject involvement with the project and provided adequate incentives to participate.
Control subjects would have had to be matched with project subjects to avoid selection
bias. This would have incurred considerable additional costs.
These issues were raised with the Reference Group and project management. Given the
problems and the costs involved and the current triangulated methodology which
explored causality qualitatively using multiple sources and methods, establishing a
comparison group was rejected.

9 CONCLUSIONS AND RECOMMENDATIONS
The evaluation has demonstrated a range of sustainable impacts for mothers, children
and individual staff engaged with the project. A cultural shift in the working practices
toward the fuller implementation of PCG and continued training in this area has
occurred across all participating centers and this has been ingrained through
developments at the policy level. This has led to a change in the responsibilities of childcare
workers who have acquired a broader range of beneficial skills in the process.
Resources and skills developed or compiled for the project continue to be utilized.
Approaches developed in the project (notably the use of video recording to help parents
and staff reflect on their practices) have also been adopted in some sites as part of ongoing
practice. Further project implementation will occur in at least one new site in
Queensland and work has commenced to explore adapting the project for Aboriginal
communities.
The main difficulties to emerge from this project were related to the issues generated
through enlisting a multidisciplinary approach to service provision and in attempting to
manage it across geographically dispersed sovereign and autonomous agencies with
independent managerial structures, differing missions and policies. In the former case
the difficulties were overcome through nurturing understanding and experiences of the
contributions and expertise available from the professional participants. A number of
strategies to enhance this have been identified. Coordinating the various sites proved a
greater challenge and one which may have been eased by the early establishment of
MOUs and documented project management/accountability procedures. However,
embedding the project in organizations with established managerial and accountability
structures would alleviate this issue.
The degree of training and capacity building achieved by the project has been substantial
and represents a considerable investment which has subsequently generated profound
outcomes for vulnerable families and their children. Clearly, the roles of the clinician and
co-facilitator are not sustainable without funding to support these positions. There have
been some moves made toward promoting the project in an attempt to secure funding at
a State level including several formal presentations of interim findings, but these have
not to date led to a continuation of the project. There is potential for expanding the
training role of centers across the sector and engaging clinicians as central to this work;
this has the possibility of acquiring funding for the role through this source. However,
the extent to which this would be sustainable, and the degree to which these activities
might impinge on the operations of an extended TtLG are unknown.
In the light of the evidence presented through this evaluation, there is an overwhelming
case to perpetuate the project in order to build on the investment and continue to
provide an intervention which has clear multiple positive impacts and sustainable
benefits for Australian families. A number of potential model refinement areas have been
identified and are currently being considered. Whilst there are areas of the service model
which may be subject to on-going context specific revision, the project demonstrates its
flexibility to adapt to and be adopted by different child center practices and contexts and
generate a range of successful outcomes for service providers and their clients.

 

For the full report with displayed data and appendicies please visit the Adelaide University Website.

 

 

The following is the summary taken from the mid interim evaluation report. Data to date suggests that the project is on the right rack and has good evidence of improved outcomes for children and their parents participating in the project.

 

SUMMARY
Background:
The Through the Looking Glass (TtLG) Project is a health, education and welfare collaborative early intervention strategy that utilizes the existing infrastructure and universality of five child care settings across Australia to intervene with families where there is an identified compromised attachment relationship between the parent and child/children.

The Project is designed to achieve specific outcomes for parents, children and child care staff. The Lady Gowrie management team worked collaboratively with the local evaluator to specify project objectives/outcomes. These were ordered hierarchically and an evaluation plan designed.

The TtLG Project provides intensive psychosocial support, therapeutic intervention and childcare as a package for high risk families in order to develop and support secure attachment relationships between mother and child. The primary target group is mothers of children aged 0-5 years. The participating families come from diverse backgrounds but all exhibit multiple risk factors including anxiety, depression and social isolation and many of the parents have reported early trauma in their own lives. There are up to seven families recruited for each of the five Centres per Wave. There are six Waves planned for the project each lasting around five months.

The TtLG project is based on Attachment Theory. The intervention draws from the ‘Circle of Security’ (COS) project model (Marvin et al., 2002) and assists parents and child care staff to understand and integrate attachment theory into practice.

Evaluation Methodology and Procedures:
This evaluation draws on Patton’s (1997) Utilization-focused Evaluation using participatory action research procedures (Wadsworth, 1998: Sankaran et al 2001).

The evaluation also adheres to the tenets of ‘Realistic evaluation’ (Pawson and Tilley 1998). The evaluation, designed and managed by the external local evaluator has utilized an Evaluation Assistant position based at Lady Gowrie, Thebarton to enhance the integration of evaluation procedures and on-going feedback into project practice. Capacity training in evaluation for all project staff has been on-going, and formal evaluation feedback has been facilitated through the Reference Group.

Evaluation data is obtained through applying a multi-facetted and methodologically triangulated approach. The approach has been flexible in order to adapt to the evolving Project and to minimize disruption to the busy workloads of those professionals approached to participate.

In consultation with Lady Gowrie Management, the local evaluators established ethically appropriate systems to collect, compile and transfer confidential data from each site to a central point for analysis. This included allocating unique case identifiers to each family, child, site and Wave in order to link the various pre and post data sets. Clinicians were engaged to help collect client data and were provided with client consent forms, evaluation information and summary evaluation sheets for each to complete.

The evaluation uses a series pre and post Project measurement tools, surveys interviews and observations to collect quantitative and qualitative data from mothers, children and TtLG staff. The evaluation toolset is attached as Appendix D. The Reference Group was requested to identify appropriated standardized instruments to measure a range of psychological and behavioural dimensions related to the project aim. No one instrument operationalised the multifaceted issues addressed and a suite of tests was subsequently adopted. This necessitated the use of video recording, external assessment (by professional assessors based in Sydney) and additional staff training. Sustained impact for targeted families over the medium term have been addressed through follow-up surveys of all mothers three months after completing the program.

Findings to Date:
The Reference Group, Project management, childcare centre directors and TtLG staff are working in partnership to develop and support the TtLG model. A range of suitable partnering agencies have engaged with the Project and are committed to it. A number of logistical issues have arisen in the implementation of the Project. The mechanisms for identifying and addressing these issues have been established through the Reference Group, on-going evaluation feedback, liaison between the sites and the Project manager and through formal training and information exchange sessions which have been well received.

The TtLG project has been very active in providing a range of capacity building activities to staff across the five project sites. This has built capacity to adopt and deliver a integrated primary care giving system, which in turn supports the TtLG families and improves attachment outcomes. This has allowed the organisation to deliver better services for targeted families and their children, (a national ‘Invest to Grow’ priority). Given the diversity of flexible responsive approaches used there is need for further evaluation of the training provided in order to identify the more important aspects and the areas which are most beneficial in order to inform the development of the training components of the Model.

Formative evaluation has revealed that mothers and fathers have been very positive about their experiences with the project and these feelings continued after completion. Mothers enjoyed the sessions provided and felt comfortable, relaxed and safe in the settings where they could freely explore their parenting and attachment issues.

Given the multifaceted and holistic approach adopted in the project model, it is difficult to identify the most important factors which facilitated improved outcomes. More than nine in ten mothers indicated that 70% of the strategies employed had helped them (with seven in ten indicating ‘a lot’) with regard to understanding their child’s attachment needs. The combinations of group and individual work with clinicians and reflections on the child/parent video films guided by insights from attachment theory and the ‘circle of security’ have clearly contributed to greater understanding of attachment. The childcare and primary care giving ethos of the centres were also highly valued.

92.4% of mothers indicated that the project had helped them to feel closer to their child, with nearly eight in ten indicating the project had helped them to feel good about themselves as parents and were more confident to look for other services and supports for their family. Nearly all mothers indicated that they had learnt more about parenting and attachment, were more confident to respond to their child’s needs, were better able to cope as a parent, felt closer to their child and acquired understanding of their child’s attachment and exploration needs. 88% of mothers noted lasting positive changes in themselves since completing the project. Around eight in ten mothers formed supportive friendships during the project with over half of the mothers engaged maintaining friendships three months after project completion.

The above findings have been supported through accounts of professional stakeholders, and are further supported by the applied pre and post standardized tools. Psychological and behavioural improvements were found to be statistically significant in nine of the eleven dimensions measured, with large effect sizes found for depression, stress and the child’s wellbeing and involvement observation ratings.

The project has improved parent competence and style and improved family functioning. Parents have increased their knowledge competence and awareness to overcome barriers to attachment, are less stressed, depressed and anxious and better able to cope as parents. Many report better parenting practices, better engagement with their children and improved child behaviours which they attribute to the project. For many parents these impacts have been sustained since leaving the project. These findings provide clear evidence that the project is addressing the national ’Invest to Grow’ priority areas of: ‘Improved family functioning’, ‘Improved parent competence and style’ and ‘Improved child social and emotional development’.

A range of issues have been identified with regard to optimising the implementation of the model. Establishing a ‘primary care giving culture’ and broader understanding of attachment theory require on-going training in a field known to have substantial staff turnover. The need to explore ways to engage more fathers, C&LD and Indigenous clients and the service implications of doing so have been identified.

There have been some issues identified which are intrinsically linked to difficulties with multi-disciplinary team working and establishing a coherent set of working, reporting and accountability procedures across the five engaged sites. The project is beginning to shift emphasis more toward formalising procedures to address these concerns in the project Manual. Similarly, the evaluation will begin to focus more on the higher order objective 5 as the project moves toward achieving its goal.