For a PDF version of this methodology that includes the full poll questions, please click on the button:
POLL: Women's rights in the Arab world
In the autumn of 2013, Thomson Reuters Foundation conducted its third annual poll of gender experts, focusing on women’s rights in Arab League states.
The perception poll of 336 specialists was designed to assess the extent to which states adhere to key provisions of the U.N. Convention on the Elimination of Discrimination Against Women (CEDAW), which most Arab League states have signed, ratified or acceded. It sought to measure how states compare for women’s rights across the broad sweep of factors covered by CEDAW, ranging from political representation and economic inclusion to reproductive rights and gender violence.
The poll produced a ranking of states – the best and worst for women’s rights – based on the methodology below.
The survey examined expert perceptions of women’s rights in all 21 member states of the Arab League: Algeria, Bahrain, Comoros, Djibouti, Egypt, Iraq, Jordan, Kuwait, Lebanon, Libya, Mauritania, Morocco, Oman, Qatar, Saudi Arabia, Somalia, Sudan, Palestinian territories, Tunisia, Yemen and United Arab Emirates.
It also included Syria, a founding member of the Arab League that was suspended by the group in November 2011.
We used CEDAW as the basis of our questionnaire.
Questions were set in six categories based on key CEDAW articles:
- Women in politics
- Women in society
- Women in the economy
- Women in the family
- Reproductive rights
- Violence against women
The poll measured sentiment across these six categories as indicated here:
- “Women in politics” refers to women’s representation or opportunity for representation in the political, civil service and state administrative spheres.
- “Women in society” looks at cultural expectations concerning women, as well as cultural factors that might prevent women from fully participating in society.
- “Women in the economy” touches on women’s power to financially sustain themselves, as well as gender discrimination in property rights and employment.
- “Women in the family” includes factors that could force a woman to accept an unwanted marriage or to discourage a woman from divorcing.
- “Reproductive rights” includes a variety of questions regarding cultural attitudes to choice in bearing children, as well as access to reproductive health care.
- “Violence against women” relates to the most dangerous forms of violence and their occurrence in each of the 22 surveyed states: trafficking, female genital mutilation, corporal punishment, marital rape and the factors encouraging violence against women.
In total, there were 36 questions, nine of which referred to details of respondents including name, age, sex, profession, employer and country of expertise.
We tested the questionnaire in-house and the average completion time was approximately 13 minutes.
Questions were designed to allow us to compile scores to rank states and leave space for respondents’ own thoughts on what they considered to be the most pressing issues for women in their countries of expertise.
The questionnaire contained:
- 14 Likert scale questions
- 6 rating scale questions
- 7 open-ended questions
- 9 questions about respondents
Likert scale questions posed statements and asked respondents to choose one of the following: highly agree, agree, neither agree nor disagree, disagree or strongly disagree.
Rating questions required respondents to rate the importance of certain factors on a scale of 1-5 (plus not applicable).
All of the poll’s six categories, with the exception of introductory questions, included both Likert and rating questions.
The “reproductive rights” category did not have any open-ended questions.
The first five categories featured two Likert scale questions each, with “violence against women” containing four.
Each category had one rating question.
Intermediate versions of the poll were tested in-house and by humanitarian, development and human rights organisations. We wanted to avoid questions that could in any way cause offense, so we asked both female and male Muslims to take the poll. We refined the questions in light of their feedback.
The final version of the questionnaire was the result of thorough research and informed by guidance from Thomson Reuters Foundation’s leadership team, the Reuters News polling team and international and national human rights groups.
We targeted local, national, regional and international humanitarian, development and human rights organisations, academics, media professionals, health care providers, refugee shelters, women’s shelters, legal advisers and activists, with a strong preference for female respondents.
We tried to avoid polling politicians and only considered those with a demonstrable interest in gender issues.
Respondents were not randomized.
The questionnaire was not posted publically online and was only sent to people who matched the criteria above. We did allow respondents to forward to colleagues with relevant expertise but asked them not to post the survey online.
We discussed cyber security issues with a top information security consultant from Front Line Defenders.
The survey was translated into French and Arabic by a professional translation firm. We created three identical surveys in SurveyMonkey, one in each language
We disseminated the questionnaire by email, pasting in a link for every one of the three-language formats.
In our emails, we offered additional security advice in case respondents felt that they could put themselves in danger by taking the poll.
For five weeks, starting in August 2013, the data team focused on distributing the questionnaire.
They ensured that at least 10 respondents answered from each state, with bigger representation for larger nations such as Egypt and Iraq.
The poll closed in the third week of September 2013.
- French and Arabic responses were downloaded from SurveyMonkey as spreadsheets and translated into English by a native Arabic speaker and by a French-speaking reporter.
- Microsoft Excel 2007 was the main software used for data analysis.
- Data was cleaned so it was in the same form and size.
- The translated French and Arabic responses were added to the main spreadsheet, next to the English survey responses, to create a master spreadsheet containing all responses from all three surveys filed in all three languages.
- Next, we filtered out all incomplete responses – for example, those that did not contain answers for compulsory questions under section 7 (violence against women).
- We then divided our analysis into two steps: Likert scale questions and rating questions. Both types were analysed country by country. We created a spreadsheet with 23 sheets - the master spreadsheet and one sheet for each of the 22 states.
- In the master spreadsheet, we filtered responses country by country and copied records into each country sheet accordingly.
- For Likert scale questions, we assigned values to scores: a negative meaning generated a high score and a positive meaning generated a low score. So the higher the score, the worse the situation for women. We used this table to run a VLookUp function on the Likert scale questions to transform statements into scores.
- For rating questions, we calculated the average score of each question.
- We then averaged all the scores – per question and per theme – for each country.
- Next, we grouped the results per category, per country. We calculated the average score for each category for each of the 22 states. We ended up with six numbers for each country (one number for each theme).
- Then we averaged the six scores, to come up with a unique number. This number constituted the final country score.
- We filtered the countries based on this score, from largest to smallest. The largest score represents the worst country for women in Arab league states and the lowest score represents the best.
- The Reuters polling team validated the methodology and ran the results by Thomson Reuters statisticians in Bangalore, India, who further validated our results.
- All questions had the same weight, as they were all based on CEDAW articles. We did not attempt to assume any relative importance to different CEDAW articles. For example, we did not try to determine whether female genital mutilation was any “better” or “worse” than marital rape as a form of violence against women.
- A perception poll represents a snapshot of the opinion of a particular group of people at a certain moment in time. We are aware that results may have been influenced by events taking place over the period the survey was conducted (August to September 2013).
- Some states had very similar scores on certain issues so it is important to recognise that final rankings may indicate only slight variations in expert perceptions.