IR 10.0 Workshop
From Sociotechwiki
This workshop will explore multidisciplinary aspects of sociotechnical research on the Internet, by providing a forum for the pooling and sharing of disciplinary perspectives, and for finding complementarity and synergy across disciplinary knowledge, methods, and approaches. Activities planned for the workshop will engage participants with theoretical and methodological approaches drawn from multiple disciplinary perspectives. Click here to see the submission that was accepted for IR 10.0.
This material has been posted on a wiki intentionally - please feel free to edit it, improve the content, add content, ask questions, etc.
Organizers of the Workshop
Purpose of the Workshop
This workshop has four major purposes. First, this workshop exists to promote awareness of and continue the community building to form the emerging Consortium for the Science of Sociotechnical Systems. Second, because sociotechnical research is almost by definition multidisciplinary, this workshop exists to explore what doing multidisciplinary research means, both for individual researchers, and for the research community as a whole. Third, there is a ‘practical’ element to this workshop regarding how to manage research from a multi-disciplinary perspective. Fourth, as an outcome of this workshop, we will create two pages on the sociotech.net wiki: one on the questions and ideas related to multidisciplinary in sociotechnical research, the other specific to sociotechnical Internet research. We will create scaffolds of these pages from the information the attendees give us prior to the workshop, update them during the days events, and receive feedback on them during the final "Discussion of the days activities".
Pre-Conference Participation
If you wish to attend this workshop, please submit the following, either via email to ifloyd2@illinois.edu or by editing the IR 10.0 Workshop Participants page on this wiki:
- A list of research questions that currently interests you - especially early-stage research questions.
- A list of phenomena in the world that currently interests you, that you would like to study. This can be combined with the above list.
- A list of theories or theoretical perspectives that you find interesting or that you currently use in their work.
- A list of methods that you use or would like to use in your work.
- One (or more) favorite reading(s) on socio-technical/social informatics or multidisciplinary research.
- One favorite reading highlighting method (either by example or description) for socio-technical/social informatics or multidisciplinary research.
(We intend to use the readings to build a core reading list relating to this activity and thus a resource for students)
If you plan on attending the workshop, please provide us with your email. We will be adding your email to a mailing list that we will use during the workshop.
If you have one, please bring your laptop to the workshop.
Schedule
9:00-9:15 am - Welcome to the Workshop
Brief overview of the purpose
- Introduce the workshop leaders.
- Introduce sociotechnical research / Working definition for this workshop (maybe some discussion of that later?)
Ask participants to introduce themselves: Name Stage in your career Department & Affiliation Brief Description of Research
- Why are you interested in attending this workshop?
- What phenomena do you study now or intend to study?
- What methods do you use?
- Do you see your work as inter- or multidisciplinary? If so, how?
- What do you hope to take away from this workshop?
9:15-9:30 am - Overview of the CSST
- We should probably present the list of all the related disciplines that was developed at the iConference workshop.
- Leave time for questions
9:30-10:30 am - Group discussion about multidisciplinarity facilitated by workshop organizers
Lead with a statement along the lines of:
"The premise of sociotechnical research is that many phenomena of interest in human activity cannot be fully understood without studying in concert the technological, geographical, and socio-cognitive components of that activity. Most established disciplines are narrow in their focus: e.g., sociology studies social interactions, cognitive science studies individual cognition, computer science studies principles of computing, etc. Given this fact, true sociotechnical research practically cries out for a multidisciplinary approach. However this is not without its challenges. While disciplinary researchers can often use established methods, theories, and theoretical approaches to study their phenomena of interest, multidisciplinary researchers typically have to develop their own methods, theories, and theoretical approaches, often by cobbling together bits and pieces from different disciplines that were never designed to fit together. What are some of the challenges in your own work that you have faced in doing multidisciplinary research?"
Then, as the discussion progresses, facilitate by asking questions along the lines of:
- "How do you balance depth of study with the breadth of the analysis that multidisciplinarity requires?"
- "How does your work fit in with the work of other researchers?"
- "Do you consciously make an effort to produce findings that others can build upon? If so, how do you go about doing so?"
- "Is your work more multidisciplinary from a methods perspective, a theoretical perspective, or both?"
- "How do you ensure that the methods you develop or knit together are valid? How do you ensure that multi-method triangulation actually produces the confirmation you seek? How do you avoid false triangulation?"
- "How do you reconcile theoretical approaches that have been developed from different epistemological perspectives?"
- Other questions?
In particular, we should ensure the conversation touches on the following issues:
- Fundamental tension in multidisciplinarity between depth and breadth of research
- Avoiding attribution errors.
- Multidisciplinarity as practiced by individual researchers
- Multidisciplinarity as a result of collaborations between researchers
- Multidisciplinarity as a collective product of the work of multiple individual research groups.
- Multidisciplinarity as a combination of research methods from different fields developed for different purposes
- Multidisciplinarity as a combination of theories from different fields/disciplines in order to better explain phenomena that cannot be fully explained from a single disciplinary perspective
10:30-11:00 am - Break
11:00-12:30 pm - Group Exercise: Developing a Research Project
We will have our participants break up into groups of 3-5 people each. Each group should have at least one person with a laptop, who should be taking notes written well enough that someone not a part of the group can read them. At the end of this exercise, the notes will be mailed out to the mailing list for the workshop.
Using the list of research questions, methods, and theoretical approaches compiled from our participant emails as inspiration, we will ask each group to pick a particular research question/phenomenon of interest, and then develop an outline for a research project to investigate it, including both methods and theory. At least one of the methods or theory should be multidisciplinary in nature. It is important to try to make this as valid a research project as possible, because it will be critiqued in the next section. The goal of the research project should be to undertake as complete an investigation of the phenomenon as possible--the projects should be realistic, but need not be simple or quick. In particular, the following considerations should be kept in mind:
- What is the primary research question to be investigated?
Given the primary question...
- What are the various sociotechnical aspects of the phenomenon being studied? For example, which of the following are relevant: human cognition, technological constraints and affordances, cultural norms, cultural knowledge, institutional policy, interpersonal relationships, etc.
- How can all the relevant sociotechnical aspects of the phenomenon be incorporated into the study that is being designed? Can they all be incorporated?
- How will unexpected features of the phenomenon being studied be discovered? For example, if a particular internet message board is being studied to see how technological affordances of the interface affect the dialog on the site, how would you discover that there are gender dynamics at play which are significantly affecting who talks and how much, independently of any interface features?
- How do the methods being used in the study effectively reveal what is happening in the world? How can we have confidence in the results?
- What are the "products" of this research? How can they be used by other researchers? How can they inform practice? How do they fit into the larger collective work of internet research/sociotechnical research? Do any of these questions actually matter to you?
- For methods: how much triangulation is enough? How do you decide that you have enough triangulation? How do you decide what methods not to include? How does this affect the results of the research?
- For theories: how many theoretical perspectives are needed to elucidate all aspects of the phenomenon? Is it possible to include them all in the research? How are decisions made about which to include, and which to exclude?
For research practice:
- What are the most important aspects to pursue – i.e., of the wealth of disciplines and methods that could be brought to bear on the problem, which is most, second, third most important in relation to the research question being pursued?
- You have 1 month to examine the issue, what do you do? You have 1 year, what do you do? You have 3 years / 5 years, what do you do?
For research publication:
- What would be the best venue for publication of this work once completed?
- Others?
12:30-1:30 pm - Lunch and Informal Conversation
Due to the economics of the conference, we will likely have to go out to lunch, and each of us pay for ourselves.
1:30-3:00 pm - Round Robin: Critiquing the Research Projects
Going around the room, we will have each group give a 5 minute or less presentation of the research project they developed. Following this presentation, we will as a group critique the project for about 10 minutes (longer if we have time), with special attention to multidisciplinary issues.
3:00-3:30 pm - Break
3:30-4:30 pm - Discussion of the day's activities, wrap up of the day
We'll have a group discussion about the day's activities, see what people learned, etc. We should probably touch on some of the points that were brought up during the introductory group discussion. We don't need to plan this very much; however, we should touch on all the areas or questions they were asked to address in designing the research.
4:30 End
6:00 Join us for dinner (own expense, place to be determined)
Possible Discussion Questions
- How can awareness of certain phenomena become a part of the analytic process of doing research without necessarily being a focus of the research (e.g., the race and gender issues from the above example of "extended theoretical articulation")? (This question is advocating an extended theoretical articulation approach for cases where it is a good idea.)
- At the same time, a researcher cannot focus primarily on phenomena outside his or her focus because they will lose sight of the research they are trying to do. So how do multidisciplinary researchers draw a balance? How do they maintain focus, yet still draw in other phenomena so they can think more critically about the work they do?
- In an ideal world, all multidisciplinary researchers taking an extended theoretical articulation approach would include all of the theoretical lenses developed in the master list. But in the real world, we don't have the time, cognitive capacity, etc. to do this work. While we might strive for interdisciplinarity and awareness of other literatures to broaden our perspectives, the primary function that this work typically serves is to keep us from overlooking obvious causes and effects of the phenomena we see, rather than to carefully, systematically review specifics. So, can we articulate the trade-offs? Can we figure out when it is appropriate/necessary to include these perspectives, and when we can leave them out of our analysis? Can we figure out ways of reducing the cognitive overhead of bringing multiple analytic lenses to research so that we can include more of them more often?
- Multidisciplinary collaboration does not end at the level of individual research projects. Internet research and the CSST are both examples of areas where researchers are interested in creating more unified understandings from work undertaken from radically different research perspectives and traditions. Therefore, can we figure out analytic frameworks for fitting together research performed by people with different perspectives so that even if we are not directly working together, our work does not end up being completely fragmented?
More on Multidisciplinarity
In this section, we present some of the theoretical motivation behind this workshop:
Multidisciplinary research takes many different forms. The most common forms are theoretical multidisciplinarity and methodological multidisciplinarity. Theoretical multidisciplinarity involves either the integration of theories from multiple disciplines, or the application of theoretical perspectives from one or more disciplines to a phenomena that is being studied in a different discipline. Similarly, methodological multidisciplinarity involves either integrating different methodologies from different disciplines to create a more robust methodological approach, or the borrowing of a methodology from one discipline and applying it to another. In this workshop, we are primarily concerned with theoretical multidisciplinarity, and in particular, with theoretical perspectives that provide analytic lenses. Our focus does not exclude theoretical constructs or theoretical phenomena, but we are primarily concerned with how these theoretical accounts can provide insight into what happens in any particular sociotechnical system being studied.
Multidisciplinary research faces a constant challenge of balancing depth and breadth. Because individual researchers have limits on time and cognitive capacity, it is often difficult for them to engage with individual disciplines as deeply as some critics would like. However, the benefit they bring is a consideration of research or analytic perspectives which are not present in a particular discipline, and which sheds new light on disciplinary understandings or practices. Different interdisciplinary researchers take different approaches to their work. There are at least three standard approaches to multidisciplinary analytic work:
- Theory application. This approach typically involves a priori adoption of one or more theories of interest. The study which follows often takes one of several forms:
- An evaluation of the theories in a particular domain, in order to establish their relevance, explanitory power, or to identify gaps in existing theoretical models. If gaps are identified, the theories are usually extended to better account for what is observed (see Wei Li's work, Vandana Singh's work).
- A number of theories are selected and then the data is evaluated to see whether or not it fits the theories that have been selected. Theories which do not fit the data are discarded as inappropriate to the situation being studied, whereas theories which do fit the data are retained as having (at least partial or potential) explanatory power. (see Nosh Contractor's work).
- Theoretical articulation. This approach typically involves a posteriori adoption of multiple theories of interest because it focuses on emergent results. Such a study often takes the form of researchers studying a particular site or activity, and then utilizing existing theory and developing new theory (when needed) based on their observations. As such, this research often uses a "grounded-theory" approach (Strauss & Corbin, Discovering Grounded Theory), or "grounded-theory inspired" approach. (There are many variations in addition to the two main schools of grounded theory.) When Strauss & Corbin were writing, there was a poverty of theory in many areas of study. Today, however, there is often some existing theory which can be used to partially explain the phenomena being studied. Thus, this approach does not preclude the use of already existing theories to explain the phenomena being observed. What it does preclude, however, is a priori selection of relevant theories. Instead, the researcher only draws on theories which are directly relevant. But finding relevant theories typically requires searching multiple literatures to pull together what is most appropriate to explain the phenomena. Furthermore, this approach is typically taken when theoretical understandings of the phenomena of study are incomplete: either poor or non-existent, or when they cover only a part of what occurs. Thus, the creation of new theory is still required to explain those observations not covered by existing theory (eg. Ribes & Finholt).
- Extended theoretical articulation. This approach involves extending the theoretical articulation approach to research by using alternative theoretical lenses which are not immediately obvious as relevant to interrogate the data as a checking mechanism for the researcher's analysis. The objective is to try and bring in as many relevant perspectives as possible (e.g., Twidale). The goal in these efforts is to supply research perspectives from many different sources so that the positive and negative aspects of each perspective can be compared and the resultant understanding of the phenomenon in question is more refined. This approach is not always appropriate, for example in studies which attempt to examine the subjective experience of a population engaged in a particular activity (in those cases, the participant categories should be used to create the theoretical account of the subjective experience). However, this approach can be a useful exercise for the researcher in an attempt to overcome known biases. A prototypical example of this, is a white, male researcher who is studying adoption patterns of cyberinfrastructure to support remote scientific collaboration. While he may be naturally sensitive to multiple community norms, boundary objects facilitating communication, negotiations for resources and power, the effects of software functionality and interface design on task-completion, etc., he may be blind to gender dynamics or to miscommunication due to different value systems brought by participants of different racial or socio-economic upbringings. Thus, examining the data he has collected using gender, race, or socio-economic class as theoretical lenses may provide him insights into what he has observed which otherwise he would never obtain.
