Cardinal Clancy Centre for Research in the Spiritual, Moral, Religious and Pastoral Dimensions of Education.
Spiritual, Moral and Religious Education is a key area for Teaching and Research in Australian Catholic University

Notes on progress in research, research supervision and thesis examination:   A compilation of ideas for the use of ACU research students and supervisory teams

 Introduction

 The more that research students and supervisors look at different research proposals and presentations of progress reports, the better perspective they develop on the research process.   They build up a broader and sharper “interpretative background”.   They become better able to discern what is happening in the development of a research project, and better able to offer constructive critical comments.   This can help those whose work is being considered.   Also, it benefits individuals in their own research programs because they can become better critics of their own work;   and better critics of their own writing.   They can learn vicariously:  both about activities that can help them;   and about pitfalls and problems they would want to avoid.   The more completed doctoral theses students look at or read, the better they become in evaluating different approaches to research.

 Research students gain from having advice from different people, and different points of view on their emerging research – and not just from their principal supervisors.   Criticism of work may be initially disheartening (or may provoke defensive or angry feelings);  but if students understand that criticism and suggestions are intended to help the research progress and improve their work (and not embarrass them), they can see critical comments as valuable and very useful.   Not all of the advice given is always useful because the individual offering advice is usually not aware of all the circumstances of the project;   but often the advice is useful to students in the shaping of their work, even if they do not follow up all suggestions.

  When in a team supervision situation for a research student, or when participating in research colloquia, academic staff can see how others evaluate a proposal (or research progress) and they can in turn sharpen their own “research discernment” capability.

  The material here is simply a compilation of the collective wisdom of ACU research supervisors and students.   It can be considered, discussed and extended to include a range of ideas that will be helpful for staff and students.   A growing sensitivity to these and related issues comes with experience.   This material does not substitute for the formal university documentation related to postgraduate research students and the work of research supervisors.

 The notes cover a number of issues related to the to the task of the research student and the role of academic supervision.   The list is neither exhaustive nor comprehensive.   It reflects the experience of ACU staff and students.   It does not cover all of the issues and questions that researchers and supervisors will encounter.   However, it provides a useful starting point for naming and thinking about the issues in a way that will help make research projects by doctoral students more successful, more satisfying, less frustrating and more enjoyable.   It should also help staff who are involved in research supervision.   The document is open to revision through additions and modifications which reflect other insights on the part of both research students and supervisors.

 If you wish to make a contribution to this “collected wisdom”, in the way of additional comments under the initial headings, qualifications to what is already there, or material under new headings, please add your writing in coloured tracking changes so that the additions / qualifications can be added to a master copy, and return the file by email attachment to Graham Rossiter at    g.rossiter@mary.acu.edu.au

 Index

 1.       General introduction to the research process                                  

 1.1       Introduction of students to the ‘culture’ of research

 There is a need to introduce new research students to the ‘culture’ of research, especially in their chosen field.   Previously, they may have listened to lectures or examined books and articles with a view of interpreting their meaning and working out how relevant the ideas were to their professional practice, and to consider practical implications.   Now they need to upgrade their focus to research.   From a research perspective, they need to become more familiar with the way that questions and issues become framed as research questions.   They need to think about how further study and empirical data collection can throw new light on the issue.   Hence, lectures and study are interpreted more in terms of research questions and what procedures might be followed to unravel the factors that are involved.   It includes critical evaluation of various items of theory and practice.   These emphases need to appear both in research-related courser work (as in EdD and in research methods courses, as well as in research colloquia.

 The other element of research culture that is important is for students to feel part of a ‘research community’ of other students and academic staff.   The research process is naturally a long and solitary one.   However, feeling part of a group which is sharing the research process with common goals is supportive.   It makes it easier for individuals to learn the skills and perspective required for a successful research study.   It provides opportunities for learning many of the ideas and skills proposed in this set of notes.   Research colloquia are particularly valuable.   They not only help orient and support students, they help build up their critical, research discernment capability.   The networking also helps students see to whom they can turn for help.   Sometimes significant help can come from other students.   The colloquia provide an opportunity for continued input about research skills, methods of analysis etc.

  1.2       Supervisors and the culture of research

 It is not only important to help research students become conscious of, and participate in, a culture of research, it is also important for academic supervisors.   Research colloquia help supervisors develop their skills in the evaluation of research proposals and progress reports  --  especially through seeing research proposals critiqued by both academic and research students.   It takes considerable time and experience to become skilled in looking critically at research proposals, noting problems with the logic and research strategies, and identifying vagueness in the definition of research questions.

  These critical skills flow over into the reading of books and academic articles in the field.   Staff and students become more skilled in identifying research questions and evaluating research strategies.   It also enhances skill in judging how appropriate are different statistical procedures for the analysis of data.

  1.3       Developing a critical perspective on research and research designs

Both research students and supervisors need as much practice as they can get in looking critically at the research proposals of students.   Where this is done in a colloquium or forum, all are able to benefit from the pooled insight of staff and students in offering critical comments about research conceptualisation, research design and method.   Skill is developed not only through trying out one’s own ideas in making an appraisal of a research proposal as well as in picking up useful insights from the appraisals made from others.

  Having to think about and consider some comments on a variety of different research topics and different research methodologies helps build up a broader perspective on the research process in education.   It also sharpens the critical skills in looking for strengths and weaknesses in concept formulation and design.

 In turn, the development of this skill in critical thinking about research has an important carry over into the students’ own work.   With more experience in appraising the progressing research of others, they can become more astute and insightful critics of their own work.   They can learn to avoid some of the pitfalls that they have seen others fall into.   They can also pick up hints and ideas for their own emerging research from what is being talked about in the process of student colloquia for the appraisal of research proposals.

2.         Issues related to the research supervision process

 2.1       Research supervision teams

 A standard in the Faculty of Education is for students to have a supervisor and assistant supervisor.   However, in some instances a research supervision team of 3 to 5 academic staff will form around the designated supervisor and assistant supervisor;   this helps academic staff get more experience of the supervision process;   it also gives the students additional people to consider their work and give advice.   A trial of these supervision teams in the School of Religious Education in 2000 has shown that all of the students concerned were in favour of the team approach, and that all of the staff concerned, including those who do not have doctoral qualifications, considered the approach valuable.   It is acknowledged that experience in research supervision is an acute problem in the Faculty of Education.   The research supervision team approach is one way of addressing the problem.   The number of people in the supervision teams can compensate to some extent for a shortage of experienced supervisors.

2.2       The level at which the research student writes

 It is the supervisor’s responsibility to let students know how they judge their academic performance.   For example, they need to let students know that their writing is at doctoral research level or whether it remains at something like Graduate Diploma level, which is more descriptive, rather than at the level of critical evaluation and a capacity to look at issues from the perspective of potential new research.

  During course work, say in the Doctor of Education program, it is important for academic staff to try and get an accurate idea of the capabilities and commitment of the students well before they get to the stage of commencing a research study.

  2.3       The importance of producing written materials in the early stage of the research process

 Both the research students and supervisors need to be careful that in the early stages of the research that the student does not do enough writing.   If they are just reading, thinking and taking notes but not writing material up on particular topics they are studying, then the work can tend to become diffuse.

 Part of successfully managing a doctoral research project involves assimilating the research culture and learning how to write and argue appropriately at this advanced level.   Otherwise there can be a danger that the student remains working at a more descriptive and less evaluative level.

 Usually in preliminary coursework as with in the Doctor of Education program, research students are required to do a considerable amount of writing on assessment tasks that should have a bearing on their thesis study.   For example, they will do work on issues within the general area of study, they will look at theoretical frameworks that may be of use in studies in that area, they would do specific work on a literature review related to their area of research.   Eventually they would also write about their proposal and research methods.

 Some of the values in this is how students can become better oriented towards the written task of a research thesis early on in the process.   They do not then drift for a long time with reading and general thinking without bringing this to bear on their thesis topic.   Also the way in which their writing is assessed should be able to help them see clearly whether or not they are performing at the level of understanding, argument and writing which is appropriate for doctoral research.

  Where an individual is not writing at a level that is sophisticated enough, it will be important for supervisors to point this out to them so they are not under an illusion that they are making good progress when this is not the case. They need to be made aware of problems and potential problems in their writing that would need to be addressed if they are to become successful in their doctoral research.

  2.4       Production of written material

 Supervisors need to insist that their students produce written material during the whole research process.   Something might be required for every supervision meeting — so that there is always written and submitted beforehand and considered by the supervisory team before the meeting.   This helps monitor the level at which the student is working.   It also provides a documentary record of the supervision process.   This will be important if there are any disputes about research supervision.   It will also be important in gradually putting together a case to convince some students that they need to discontinue the study.

Some supervisors have a practice where the students have to write a brief summary/response to the supervision meeting highlighting clearly what the issues were and what particular tasks or questions they needed to address in the next phase of their work.   This also helps as a record of the supervision process.   This can help students who tend to ‘drift’ along at an unsatisfactory level.

  2.5       The ‘honesty’ and the ‘devil’s advocate’ role of the supervisors

 Supervisors need to make it clear to the students at the beginning of supervision that they will always be honest with their opinion and judgments.   Because their role is to be that of a devil’s advocate, that is, not to let anything get through which is below standard or which would be called into question by a thesis examiner.   Some of their comments about the students’ work may be felt initially as a personal blow or a humiliation or a devaluation of their work.   The supervisors need to periodically emphasise that their critical comments are ‘professional’ and related to the students’ work  --  they are not ‘personal’ comments.   The supervisor’s role is to help the students become more critical of their own work, their own logic and their own writing.

  2.6       Development of writing skills

 Often one of the most crucial roles of the supervisor is to help the students learn how to write with precision, clarity and logic that is appropriate for a research thesis.   Students need to know if their supervisors think that they are not yet at this level.   Part of the process of developing this skill in research students is careful editing and feedback on their written work.   At times, this will even get down to questions of grammar, sentence construction and the meaning of words!   Students need to learn that they can take nothing for granted in terms of their logic and style;   they have to prove that they can write at the level that would lead to a successful doctoral thesis.

 2.7       Students who may be struggling or who may not have the ability to follow through to completion of a doctoral thesis

 Where a student is encountering difficulty and is not working or writing at the level that is regarded as satisfactory for a doctoral thesis, then they should not be left in any doubt about this by their supervisor.   They will need to understand that in the supervisor’s opinions, “they are not yet working at a doctoral research level”.   This would be the case if their writing was just descriptive, if it lacked a level of interpretation and evaluation that is taken for granted at doctoral research level.

  2.8       Change in the relationship between student and supervisor(s)

 At the commencement of the supervision of a thesis, particularly where the student is working out his/her research proposal and research methodology, the supervisors will probably know more about the topic than the student.   However, if the research progresses satisfactorily, then eventually the student should become much more of an expert in the topic and should know much more about it than the supervisors.   Hence, the role of the supervisors will change.

  The devil’s advocate role of the supervisors is important to make sure that nothing is put down in writing that could be seriously questioned by a competent examiner;  similarly, for a research procedure.   The supervisors’ role, as noted before, is to help the students who become self-critical in a constructive way.

  2.9       Conflicting views of what should be done on the part of supervisor and assistant supervisor (or research supervision team)

 It can be expected in the normal academic situation that academic staff in the same field will have different professional views about practices and procedures in the conduct of research.   This could lead to different interpretations of what is the best track for a research student to make.   At initial colloquium presentations, a research student can learn how to appraise different suggestions and ideas made by the group (including academic staff and other research students) on what might be the best approach for them or on what might be some particular pitfalls or problems that they should try to avoid.   Ultimately the research student and supervisors (and supervisory team) will need to appraise and decide just which suggestions might be useful to follow.   In practice, some are useful, some are not.

However, the situation is more difficult when there is a professional difference in opinion about what should be done on the part of supervisor and assistant supervisor.   Hopefully, it could be expected that they could work together cooperatively, while allowing for some differences of opinions.   However, if the differences of opinion are too great, this could put the research student in a difficult situation by being pushed in different and conflicting directions by the different supervisors.   If this is the case, something would have to be done to change the composition of the research supervision team.

  Usually when students have some choice about the supervisor and assistant supervisor they get and where they know in advance the areas of expertise of these academic staff, possibilities of conflict between supervisors are minimised.   However, the possibilities for conflict need to be considered.   The research students need to speak up if there is a situation where they feel they are caught up in conflict between their supervisors.   It is also important for supervisors to understand that any such conflict can be very damaging to their research students.   Therefore, they need to be careful that anything like this does not really become a problem for their particular research students.

  An example of a problem might be where one supervisor is pushing for an empirical study involving quantitative data from questionnaires whereas the other supervisor might be pushing for a detailed ethnographic study.

  Usually, if the students are capable enough of doing a research degree at doctoral level and if they have a reasonable introduction to research in their area of interest and an introduction to research methods, then the particular research task they take up and the methodology they adopt are more likely to be one of their choosing rather than one which is imposed or advised by supervisors.

  Another example of a potential problem:   This is where different supervisors appraise the situation of a student’s developing thesis with different interpretations.   Each supervisor may be convinced that their academic colleague is recommending something that is inappropriate and which will be harmful for the student’s thesis.   This conflict needs to be resolved by the supervisors themselves.   It should not be worked out in front of the student.   Supervisors should not try to influence the student not to follow the proposed direction suggested by their colleague.

  2.10     Getting the thesis topic and research questions clarified as early as possible within the higher degree research process.

 It takes time doctoral students to work out their particular thesis topic and delineate it in such a way that it is a realistic and manageable project and not something that is vague or too large to be covered.   It takes time and a number of reviews of the topic and proposed research questions before a final title and research questions emerge.   Students and their supervisors need to be aware that this should not take too long.   For example, the topic and research questions should normally be resolved and articulated during the appropriate parts of coursework in the Doctor of Education program or for a PhD student, precise topic, research questions and a thesis proposal should emerge within say a year and a half of part time work.

  It sometimes takes research students a long time to sort out their research topic because they are reading widely and thoroughly and they are coming to terms with the new knowledge and the broader perspective in their field that they have developed through this reading.   As long as the students know that this general reading process should not go on indefinitely, by some appropriate stage, the student should be able to delineate the particular area where they plan to locate their research.

3.       Some principles and issues arising from experience with the examination of doctoral theses

 3.1       The title of the thesis needs to reflect accurately the work that has been done

 The title of a thesis needs to be clear and it should inform the reader precisely on what the study is about.   If it is vague or misleading, it can give examiners expectations of what to look for which will not be fulfilled  --  conflict between expectations and reality.   An example, a thesis which had the words “Christian education” in its title, did not at any stage define that term, and did not include any references to this area (Eg.  Journal of Christian Education, and books with those words in the title).   This made examiners think that the researcher did not know what he/she was writing about or had not read widely enough to locate the research within a suitable framework.   A researcher cannot presume that readers will share their personal perspective on a topic.

  3.2       All major parts of a thesis need to be adequately reflected in the review of literature

 If a thesis, in either its title or in its research questions, suggests that A, B and C etc. are important components, then the thesis will have to show that it has attended carefully to the research literature on A, B and C.   Linkage with the literature is a crucial part of the research process to show where the researcher is ‘picking up’ or ‘filling in lacunae’.   Eg.  a thesis looks at relationships between science and religion, but makes no reference to the significant literature that exists in that area.   If a thesis covers a major area in its title, then an examiner would expect that the researcher shows a familiarity with the research literature and other doctoral theses in that area.

  3.3       Need for an adequate philosophical basis for a thesis with empirical studies

 Some theses which are primarily concerned with reporting empirical studies may be flawed by a lack of philosophical clarification of the key concepts.   In other words, there is a major problem with validity;   what is measured may not really be what it is supposed to be.   For example:  comparisons of two articles concerned with ‘measurements’ of ‘religious identity’.   One article measured Catholic identity in school students using a twelve point scale with simple questions like:   “Do you think religious sisters are good people?”   “Do you think that going to mass is good for Catholics?” etc.   No consideration was given to what constituted religious identity and how it might develop.   Still, the article claimed that the scale for which it was providing validity and reliability could be used for an analysis of identity development of school students and for evaluating the effectiveness of the ethos of Catholic schools.   By comparison, another article included seven pages outlining the philosophical basis for the concept ‘religious identity’ before it reported empirical data on teachers’ perceptions of their aims with respect to the communication of religious identity to pupils.

  3.4       A problem where the following of particular statistical procedures seemed to be more important than the quality and validity of the data

 Sometimes a researcher seems to base the ‘weight’ of the thesis on elaborate statistical procedures while having major flaws in defining what is being measured.   This can occur when special statistical procedures are used without adequate philosophical foundations  --  again, problems with validity.   No amount of “reliability” statistics will compensate for a lack of “validity”.   It does not help if the statistical procedures are elaborate if there has not been sufficient clarification of what is being measured.

  3.5       Logical sequence to concepts and processes in the thesis

 A thesis needs to flow naturally from key concept to key concept where this sequence is needed to explain what is being done and how the research questions are to be addressed.

  For example, the second article noted above about religious identity wanted to examine education concerned with the communication of a particular religious identity.   In the clarification of concepts, it began with the notion of identity.   Then it looked at the concept religion.   Then it looked at frameworks or particular theories for relating identity and religion.   This lead to a notion of religious identity.   Then it moved towards education concerned with identity and education concerned with religious identity.   It selected a framework that the researcher thought would be most appropriate for addressing the research questions.

  In this study, the researchers specified particular frameworks for their concepts;  Eg. Paul Ricoeur’s narrative theory of identity (as opposed to other theories), A ‘structural’ theory of religion (by contrast with other theories etc.)   The researchers also showed why they made these choices – advantages and disadvantages etc.

  3.6       The thesis needs to “tell the story” of the research work in a logical sequence

 The sequence that the researcher is going to follow in telling the research story needs to be made clear at the very start of the thesis so that the reader can see where the whole project is going.   Eg.  The reader can then know in advance that one chapter would look at key definitions;   another at key definitions;  another at the literature;  and so on for methodology, results and discussions.   If the work on one particular area goes into detail, it is sometimes difficult for the reader to remember where the thesis is going  —  particularly if it is being read on and off for a period of time.   The reader may need to go back to the start, or to some page, where the reader knows there is an outline of the sequence.   Sometimes ‘connecting phrases’ or ‘introductions’ to chapters or ‘summarising conclusions’ can be used to give the reader directions as to where the thesis is going next.   This can help the reader remember that this step needs to be completed before the next one can be taken up appropriately.   This sense of logic and sequential development of the thesis are important for giving the thesis a sense of coherence and  unity —  that it is, one integrated study, where all of the different sub-projects come together to form an integrated whole.

  3.7       Thesis a coherent whole and not an aggregation of unrelated sub-projects

 The sense of cohesion and integration (noted above) is also strengthened by the way that the researcher argues the case as to why this step was needed and why it may have been necessary before the next step as well as how it is important for the whole study.   If the various component projects within the overall study are not well justified and if they are not coherently related, examiners may get a sense that the thesis is an aggregation of a number of relatively independent and possibly arbitrary studies that are not always clearly connected with the main argument.   For example, a number of theses in Religious Education will include sections on Church documents, on historical approaches to Religious Education, theological perspectives, developmental theorists etc. without clearly justifying why these components are required in the study.   If the case of how and why components are relevant to the whole is not well established, examiners can be given the impression that unjustified elements have been included.   When this happens, the thesis can look more like a series of Masters degree assignments put end on end, rather than a systematic, coherent thesis which clearly has progression from sub-project to sub-project.

3.8       Language style

 The language style in theses comes from an academic research culture.   Prose should be objective and ‘matter of fact’.   First person should be avoided with the exception of particular places where the writer is talking about his/her own particular viewpoint.

  An objective style can be difficult for some researchers who want their language to reflect their passion for their work.   They may feel that unless their views are expressed passionately, their project is compromised.   This is not the case.   A good thesis is one where the data and arguments presented can stand up for themselves.   The writer should avoid emotive language and should be careful about the use of words like “very”, “always”, “sometimes”, “significant”, etc.   The language should also avoid journalistic words like “reveals”, “exciting”, “revolutionary”, “wonderful”, “fascinating” etc.

  3.9       Avoid first person and present tense

 Avoid first person and present tense.   Keep to third person and past tense.   It also helps to avoid stilted phrases like the point of view of this researcher is in general, avoid first person and present tense.

  3.10     An “invisible” style of writing

 The best style of writing is one that is “invisible”.   This means that the reader can have all of the information and arguments made available transparently.   The writer’s style does not ‘get in the way’.   Peculiarities in the writers’ style are eliminated and these do not distract the reader from the central argument.   It always spells trouble for a thesis when an examiner has to stop at the end of the sentence or paragraph and ask “What does the writer really mean here?”   Once this uncertainty occurs, the thesis becomes more difficult to read;   the examiner becomes increasingly unsure of what the author intends;   the ambiguity in meaning compromises its integrity as good academic writing.

3.11     Avoid “journalistic” style, also exhortations and ‘sermons’

 Writers of a thesis should avoid a journalistic style.   Some writers also fall into the trap of writing in a “sermonising” or “exhortatory” mode.   This shows up in the use of words like “we should”  “this must be done”  “this needs to be covered” etc.   Researchers need to have their data and arguments clear and conclusions can be naturally strong.   But, their strength is not enhanced by emotive language.

  3.12     Arguments need to be clear and well organised logically in sequential paragraphs

 Writers should not have any more than one or two key ideas in each paragraph.   If paragraphs get too overloaded with different ideas, it becomes difficult for the reader and the overall coherence of the argument suffers.

3.13     The use of headings

 The argument in a thesis can be enhanced by a systematic use of headings which highlight the logical structure of the argument.   They also serve as important signposts for the examiner and other readers who can be helped to see what the focus is in a particular paragraph or section and this can help them see more clearly where this fits into the overall project.

  Thesis writers need to avoid the problem which can occur when they put too many ideas into paragraphs and have the paragraphs unrelated and not part of a natural sequence.   Where this happens, the reader feels bombarded by a lot of unconnected, aggregated data and ideas that are not organised with some overall thrust.   Where this occurs, the reader senses a weak argument or that case that has been sufficiently developed.

  3.14     Too much detail in reporting data --  especially background data

 Data, particularly background data to the study, needs to be reported in as concise a way as possible.   It is perceived as burdensome by an examiner to have to wade through a verbose description of background data when this might have been summarised very efficiently in a table.   As a general rule, data should be presented as clearly and as precisely as possible with the follow up opportunity to pinpoint the important findings and later to discuss their meaning.

  3.15     Sequence for ‘Data’ and ‘Discussion’ and relationship between results and discussion/implications

 In general, the scientific sequence for the presentation of research, (which has had a significant influence on the quest by research in the social sciences for scientific respectability!) should be followed  —  this old scientific sequence is introduction, materials and methods, results, discussion.   The question arises as to what is the most appropriate place for the discussion of the meaning of the data and of interrelationships with the findings in the literature (as well as for suggestion of implications).   In general, the discussion should follow the section where the data is reported and key findings highlighted.   If discussion is continually mixed in with the data, examiners and readers can find the boundaries between the results and the writers interpretation too ill defined.   It becomes difficult to differentiate between “data” and “discussion of its meaning”.   When this happens, readers can lose sight of what is “reproducible data” discovered in the study and what is the interpretation of the researcher.   It should be possible for a reader to see clearly what the data is and perhaps come up with a different interpretation from that supplied by the author.

  Another problem is where the researcher states a view and uses data to support the view.   Rather than present the data clearly and then give an interpretation of its meaning and significance.

A general rule — have discussion follow the results.   This may be done after all of the results are presented.   Or at times, chapters might include a significant amount of data followed by discussion;   this is followed by the next chapter with another major area of data followed by discussion.   Where this happens, there is a need for some overall chapter at the end where the interpretation and meaning of the study is rounded off, tied up and where implications for theory and practice and further research are noted.

Return to the top of the document

Return to Cardinal Clancy Research Centre