Talk:Opinion polling on Scottish independence

FocalData Poll October

Was doing some tidy up work on links to data, really struggling to find data tables from this Polling Company, yes I can find published data from the client but we should be linking to best data. Focaldata appear to require an account to be able to access their published polls, which seemed a bit off. Anyone else having any luck with this? Soosider3 (talk) 11:59, 27 November 2023 (UTC)

Looking at the data published on the clients site and looking at the partial tables they published, it is difficult to see where the numbers for both the Yes/No question and the Leave/Remain, they appear to be subsets of about 500 responses and not the 1000+ claimed, this also raises the issue of reporting on what are actually subsets of a full poll.

Information can be found here

clients article https://www.these-islands.co.uk/publications/i389/its_the_way_you_ask_them.aspx

clients published tables on google docs https://docs.google.com/spreadsheets/d/1kkmgMPSSDAOfIgwjowGKJr6aXjowJaGc6fnZaP_O_IE/edit#gid=0

Please have a look and see what you think, to my mind there is something not right here, 4 weeks later and no pollster published data, sub sets masquerading as full reports. Soosider3 (talk) 14:20, 27 November 2023 (UTC)

I had emailed focaldata and got response pointing me to the google docs link, is this normal for a pollster? Have responded asking for more clarification especially around querying if this is their usual practice and why so many questions appear to be missing from the published information. Soosider3 (talk) 19:21, 27 November 2023 (UTC)

That's a couple of days now and no comment, should I take it that silence is agreement and remove this polling data? Soosider3 (talk) 14:43, 29 November 2023 (UTC)

Focaldata are a member of the BPC so their polling should be included in the article. One of the stipulations of being a member of the BPC is that the polling data should be published within 2 working days on their website. AlloDoon (talk) 10:21, 6 December 2023 (UTC)

Good morning, still not able to find data on their website. Any thoughts on data as published by client. These appear to be subsets but unlike previous smaller sample polls no indication that they have weighed for smaller sample, if correct this has to raise questions about the inclusion of this particular poll Soosider3 (talk) 10:39, 6 December 2023 (UTC)

It is acceptable for Focaldata to link to Google docs so long as it contains relevant polling data and sub-sampling. The weighted sample of the poll which you have linked on the independence question is 545 for both Yes/No and Remain/Leave questions. As this sample is weighted and within an appropriate margin of error below 7% it is acceptable to the poll listed in the article with a note explaining the sample size is smaller than 1,000 and has a margin of error above 3% AlloDoon (talk) 16:01, 6 December 2023 (UTC)

The point I was attempting to make was, if the link is not from the pollsters website can we be certain of its veracity, at the very least it raises the suspicion of a doubt. BPC rules very specific re publishing on pollster website. Until now all links to data tables have been from pollsters website and thereby can be assumed that they are the tables the pollsters stand by.

Re numbers of samples, not convinced that these are other than subsets of a full poll, I see no evidence of them weighting for smaller sample or generating what margin of error would now be. Its a question not a statement I am making, where is the evidence to support them being treated as proper polls, all be it small ones. Soosider3 (talk) 11:44, 7 December 2023 (UTC)

It getting on 2 weeks since I first posted my queries about this poll, in particular issue of link to data tables and use of subsets.

I am minded to remove them until we can establish there usability Soosider3 (talk) 10:16, 10 December 2023 (UTC)

This from 'theseislands' article is an amendment from the original publication, it offers an explanation as to how the numbers were arrived at, in essence they ask the same question twice at different parts of the interview then combine those numbers. Once eraly on and again later in process, there is no publication of all the questions between those referred to. This is unacceptable, as I am sure you realise that questioning order can and does have an impact on returns, to then merge these 2 separate and different responses to masquerade as a single questioning is highly dubious and to present as one response is misleading. I intend to delete Focaldata information until those who might support there inclusion can answer those queries. Soosider3 (talk) 13:42, 14 December 2023 (UTC)

I don't accept this at all, they are BPC members, not you. I'm going to revert these changes.

Your incessant determination to remove any poll which doesn't support your political agenda is abusive in the extreme, and needs to stop. RERTwiki (talk) 16:44, 14 December 2023 (UTC)

The assertion that about my motives do you no credit. Look at the data posted and try to square it with what was published by the client. The client stated in their article that they had asked the question twice as q6 and again as q24, effectively creating 2 subsets and then amalgamating them, at best highly questionable activity. I first raised my doubts over 2 weeks ago, but it has taken my action to garner a response. At present the pollster has not yet published the data on their website ( contrary to box rules) By all means explain if my rationale is faulty, I have always understood we don't publish subsets adding 2 together does not make it okay. I would remind you that not only the wording of questions but the order they are asked can impact results Soosider3 (talk) 17:09, 14 December 2023 (UTC)

The procedure specified in the InTheseIslands article is fine. They asked everyone both questions, and in order to avoid the bias implicit in, say, asking yes/no first and remain/leave second, they split the population into two and asked each in different orders. This seems eminently sensible to me, and obviously they have thought about the problem. They are the professionals here.

One can make a reasonable argument that visibility of both yes/no and remain/leave framing is what people would see in an actual campaign. For certain there will be a campaign for "Yes!" and the unionists would have to be blind and stupid not to campaign on "Remain". That is, asking both questions plausibly creates a more realistic response.

We are not quality control for the polling industry. This poll is good enough for What Scotland Thinks, and is produced by a BPC member. We should have a strong bias in favour of inclusion, for reasons previously cited. The poll should stay in the list.

Yes, the poll favours NO, but by about the same margin that MORI always favours YES and much less than the outlandish margins FindOutNow has favoured YES in some polls.

I don't appear to have the same conception of credit as you. As my grandmother used to say, least said soonest mended. RERTwiki (talk) 10:21, 15 December 2023 (UTC)

Your grandmother sounds a very wise person.

I would respond by pointing out that it is not the pollster offering these explanations but rather the client, that the pollster has not issue these tables (as of yet) and there is no explanation of methodology from the pollster. We should always be cautious about sources of information and how much weight to give them and would suggest that a campaign group maybe somewhat less reliable than the actual pollsters ( who are missing from this discussion)

Please note that it is well established in polling circles that when the question is asked in the polling questionnaire can impact its result, just as the questions asked leading to the question can also impact the result, we have no data as to what Questions 11 to 24 might have been.

As I say there is some highly questionable stuff happening here, as well as not polling 16/17 years old or weighing for likelihood to vote, we have (to the best of my knowledge) the Q being asked twice at different stages of questionnaire.

This is unusual practice and should be noted as such, perhaps we should include a separate line for each of the small polls with the appropriate information in the notes field, as if not subsets they are small polls

Notable that yet again we have a mention of the "methodology bias" we had some discussion on here earlier in the year around Findoutnow poll, I had did a wee bit of research on that and think there maybe a link between what pollsters weigh for in previous voting (ie Indyref, Brexit, Last GE and last Holyrood) and some of the variation. Perhaps we should have 4 wee columns to highlight this with an x if weighed to that measure Soosider3 (talk) 12:52, 15 December 2023 (UTC)

I don’t think 4 more columns is a good idea.

I’ve seen polls before in other contexts that randomise question orders for the reasons you mention. This is the same. I think we just have to take the poll at face value.

The links allow really keen readers to form their own opinions about polls.

If you want to add a brief note in the usual place highlighting what they have done I think that’s entirely reasonable. RERTwiki (talk) 10:13, 18 December 2023 (UTC)

Fully in agreement with RERTwiki here. Lets avoid any confirmation bias. AlloDoon (talk) 10:50, 20 December 2023 (UTC)

Oh behave yourself, lets report the data as it is presented in data tables and not as per client. Soosider3 (talk) 12:15, 20 December 2023 (UTC)

Aye, perhaps a note is the way to go at least we seem to have some consensus on that, I am aware of the asking the questions in a random order process, however that is not what happened here, where the same Q was asked at 2 different stages, in effect producing 2 small polls (at best) perhaps we should report them as reported by data tables ie as 2 polls rather than as client presented them, that is usually our default position - to go to the tables as most reliable source of data, information clearly presented in a manner that is obvious to the reader, rather than how the client would prefer it. Otherwise we are actually in danger of manipulating the data, check out the data tables pollster has presented them as seperate that is how we should record them

I see that Focaldata have at last published the tables on their website and it only took a complaint to BPC to get it done, perhaps not a ringing endorsement of them as political pollsters. Still no statement from them as to methodology used. Soosider3 (talk) 12:13, 20 December 2023 (UTC)

IPSOS poll removal

Happy New Year!!

I noticed when updating the charts that two IPSOS polls from back in '22 have been removed from the main table.

It doesn't make any difference to the charts, as they were already marked as non-standard questions and have never been included. I noticed they were missing when I didn't have to mark them for removal.

There is a comment in the edit history to the effect that they didn't ask about a future referendum, which seems quite unlikely, but might be grounds for excluding them.

I'm generally very negative on memory-holing interesting data, and would rather this was back as it was.

What do people think? RERTwiki (talk) 11:53, 17 January 2024 (UTC)

and a happy new year to you.

That's an odd one as notes field had already referred to it. Given length of time polls had been present I would assume our more regular editors were happy enough, therefore have reverted them. Soosider3 (talk) 12:18, 18 January 2024 (UTC)

'Additional Citations for Verification'

Eh? Anybody any idea what this is about?

Since essentially every poll is cited by a link to the originator, this makes little sense to me. RERTwiki (talk) 17:34, 30 March 2024 (UTC)

Absolutely agree with you, feels as if only the text has been read without reading and understanding the actual poll figures. Happy enough to remove as cant see way to actually engage otherwise. Soosider3 (talk) 13:18, 31 March 2024 (UTC)

Well, it's gone, I'm happy, thanks! RERTwiki (talk) 17:18, 9 April 2024 (UTC)

Having to suspend graphs

Hi - I’m temporally capacitated in producing the charts, potentially for a long time.

I think we need someone to work on the charts in similar vein. The current work to do by this largely on the unionist side makes a sensible approach when the majority of work on items is SnP.

Please reply if you are interested. RERTwiki (talk) 14:49, 19 April 2024 (UTC)

Hi RERTwiki, really sorry to hear that you are not able to continue to produce graphs, many thanks for all the time and effort you have given to this task, I for one really appreciate the effort taken.

Would be happy to take a look at what is involved before fully committing myself to task. Not sure how we would progress that but am open to suggestions. Soosider3 (talk) 10:15, 23 April 2024 (UTC)

Some how I ought to be able to supply all interested the source file of these graphs, which is an excel file, though someone needs to advise me how to. How do I get you (and others who require ) my email?

Still believe that people with different views maintaining data and graphs is a help. You getting the data in and me getting the graphs in is helpful, but que sera sera. RERTwiki (talk) 15:35, 25 April 2024 (UTC)

morning

had a look at how you might privately transfer data, looked at a few data sharing sites but they seem a bit of a palaver.

Simplest method might be to create a gmail account that you can delete once exercise completed

I have created one to receive files its indypolling@gmail.com, please feel free to send files I will acknowledge when recieved Soosider3 (talk) 07:24, 26 April 2024 (UTC)

I’ll send something asap, please can you make sure all

interested can seen the thing I send. RERTwiki (talk) 09:09, 26 April 2024 (UTC)

You can be absolutely sure that I will share file with any and all interested parties Soosider3 (talk) 15:36, 26 April 2024 (UTC)

Hello all, I have received the files from RERTwiki, bit of a saga as they went directly to spam and I've only just noticed them.

As promised I am happy to share the spreadsheet with any and all interested parties please just drop me a note at the above email indypolling@gmail.com

In the meantime a quick glance at the spreadsheet is testimony to just how much work RERTwiki has done on these over the years and we all owe him deep gratitude for his efforts.

I will explore the spreadsheet over the next few days and have a go at producing the graphs as per RERTwiki, it may be a good time once we are more familiar with the spreadsheet to have a discussion on how we see it developing, so any ideas would be warmly welcome. Soosider3 (talk) 11:44, 14 May 2024 (UTC)

YouGov polls

It would appear as if yougov has changed its presentation of data over the last 6/7 months in that what we have been presenting as Undecided for you gov includes Refused and will not vote, resulting in an apparent increase in Undecideds and possible reducing return for both No and Yes I would welcome others views as to what (if anything) we might do about this, think the options are 1 Do nothing, continue to display information as produced by YouGov, perhaps include a note as caution. Does that mean we should not include them in any graphs as not directly comparable with other pollsters 2 We remove Refused and Wont vote and leave the data as it is with it not totalling 100% 3 We remove Refused and Will not vote and adjust to ensure total is 100%

Having on previous occasions argued against authors re calculating data, I personally believe that in this particular circumstance we should do so and would purpose we do so for any polls presented in this manner. YouGov poll of 13 to 17th May is recorded as 38/45/10 adjusted to these 3 making 100% it would show as 41/48/11, making it consistent with other pollsters and comparable in graphs

I would welcome thoughts on this Soosider3 (talk) 08:53, 17 June 2024 (UTC)

I agree, for consistency it makes sense to recalculate. AlloDoon (talk) 11:16, 16 August 2024 (UTC)

Okay going to give this a go with latest yougov poll, please check my calculations a snot confident on this Apgup (talk) 13:40, 17 September 2024 (UTC)

Here are my calculations and my workings

Deducting Would not vote of 6% and Refused of 2% brings total down to 92% who are either Yes, No or Undecided

Using that ratio we get Yes at (37/92)*100 = 40%

No at (47/92)*100 = 51%

Undecided at (8/92)100 = 9%

If that seems okay I will amend entry and include note to show we have recalculated.

Somewhat annoyingly Yougov appear not to have asked about likelihood to vote Apgup (talk) 13:52, 17 September 2024 (UTC)

Graphs

It is now getting close to a year since graphs were last updated and we should be having a discussion about removing them at least for the moment. Had looked at the work previously carried out by RERTwiki and have been unable to get them to work, they apparently require macros that I do not have. We perhaps need at same time to have a discussion about what we want the Graphs to present, polling for Independence is unlike most other political polling in that it does not have specific start and end dates, additionally the comparatively few polls carried out present a challenge as trends. I have carried out a review of polling and present my findings hopefully as a way to inform discussion. In the 10 years since the Independence Referendum there have been 281 polls carried out Since Jan 2015 These tend to come in irregularly

Pollster	2015	2016	2017	2018	2019	2020	2021	2022	2023	2024	Grand Total
Norstat/Panelbase	4	5	6	5	6	8	10	6	5	6	61
Survation	6	8	8	5	3	5	8	1	7	4	55
YouGov	7	8	5	3	3	4	4	5	10	6	55
Savanta						3	13	7	4	2	29
Ipsos	1	2	2	1	1	2	4	4	2	2	21
Redfield & Wilton							2	1	10	7	20
FindOutNow							1	1	5	1	8
BMG		1	3				2	1			7
Opinium							5		1	1	7
Lord Ashcroft					1		1		1		3
Stack Data							2				2
TNS	1	1									2
Techne								1	1		2
Hanbury							2				2
Deltapoll				1				1			2
ICM	1										1
Focaldata									1		1
Kantar			1								1
More in Common										1	1
JL Partners						1					1
Grand Total	20	25	25	15	14	23	54	28	47	30	281

Of these 281 polls 241 have been produced by the top 6 contributing pollsters and 171 by the top 3.

We need a small number of graphs but ones that perhaps better reflect the actuality of Indy Polling

I would suggest we look at

1 Graph showing Yes/No/Undecided back to 2014

2 A briefer Graph perhaps covering last 24 months

3 If possible perhaps a graph to compare at least the top 6 contributing pollsters

I would welcome thoughts and ideas Apgup (talk) 12:28, 21 October 2024 (UTC)