Direct link: https://qsurvey.mozilla.com/s3/413b49c8ae9e
@neme loaded questions are loaded.
The “Want most” to “Want least” scale is loaded AF.
Where is the option for “I don’t want any of these things”?
Edit: Yeah, fuck that. That survey is bullshit. I stopped bothering to give answers due to the multi-choice questions seeming like a way for Mozilla to have a wank about itself.
This is fairly standard survey design, I believe. They’re not looking to know which features are wanted in general; they want to know their relative popularity. The sets you’re presented are randomised (i.e. we don’t all get to see the same sets), which allows them to get a ranked list of lots of potential features, while only having to run ten survey questions per participant.
If you get a set with three features that everyone likes or dislikes at about the same level, then it doesn’t really matter want you answer: they’ll all end up at the top or bottom of the list, respectively. Because each of those options also get presented as part of different sets to different users, where different answers can win out.
@Vincent couldn’t finish the survey purely because of the questions suggesting that I should “want” something.
Perhaps if they asked the question differently, they’d have gotten a completed survey from me.
I can’t answer loaded questions.
The samples they get are meaningless if only people who complete the survey are counted.
The fact that I couldn’t select none of them and move forward, meant something: Jerk Mozilla off, or don’t.
I chose not to, and I am a Mozilla user!
I’m half-way through the survey right now; and rather than continuing, just stalling because I don’t want to rank another set of three options that I don’t care about. Some of the choices already given were like “well, I guess I’ll pick the feature that I’ve at least thought about using once…” but now it’s just a list of 3 things that I don’t want whatsoever. I’m trying to give useful feedback, but I feel like I’m really just giving noise.
The problem with this design is, if people do not care, then they will give random answers, if they don’t have the option to not care. Also this would be important information for Mozilla too, if many people do not care about a specific question. So I feel like they should have done that. But, who am I…
Presumably if people don’t care, they don’t fill in the survey. But as an extra failsafe, they’ve also included the feature “twice as slow as your current browser”. If you rank that high, then your result can probably be discarded.
But yeah, this design has worked well for many other surveys, so presumably it’ll work well for this one. They’re the experts :)
You’re bang on. It’s called MaxDiff. I use it frequently in my line of work to prioritise product or service messaging with panel data. It’s better in some cases to use Inferred preference rather than stated, but generally good to keep the options comparable in “size” of offer.
I would never interpret a MaxDiff model low end result as “wow, 5% of people want slower browsers.” Instead I’m focusing on the top cluster. As with any model, they’re only ever so accurate. Don’t read into the questions too much.
Why not just get one big list with like 4 answers:
- really want
- want
- meh
- don’t want
How is that worse than getting like 10 screens of relative answers?
Because you’ll end up with ten features that all have overwhelmingly “really want” and “want” answers, and then you still don’t know which of those ten to work on first.
I don’t know if the survey questions are loaded, but it feels like they could easily be misinterpreted.
For example, somebody might rank the “organize toolbar buttons and AI chatbots” even if they hate AI’s snake oil, and now Mozilla has a data point where they can say “Some of our respondents said they want AI as much as side tabs!”
This seems especially sketchy when the side tab idea came directly from a vocal portion of Mozilla users, while the decision to follow the AI chatbot trend was decided by the same management that overpays their CEO every year.
Which of the following attributes would you most want your new browser to have, and which would you want least?
Twice as slow as your current browser
Is that a joke?
For me that was together with
A privacy-respecting AI assistant that makes your browser smarter by learning how you use it
So I don’t really care how slow the browser is, as long as it doesn’t have an AI “assistant” that is monitoring my browser usage
But why? If you want that, you can just have it outside of your browser. Or maybe get an extension that works with an AI assistant on your machine.
I honestly don’t care either way about an AI assistant. I don’t intend to use it, so I’d much rather their efforts be spent elsewhere.
Yup, I stuck that as “least want.” I already marked “2x faster performance” as “most want” on another question, so hopefully it all shakes out in the end.
You clearly missed the point about “privacy respecting.” It will only share data with Meta, Google and the US government.
Wait, I swear mine said twice as fast. Well I guess I got filtered then. Lol
Makes sense. What about those who click that option as a joke? Maybe discount all other replies from that person because of that too?
lol, yeah. 👍
I was doing a political poll just the other day and the third or fourth question was a color question like: “Which of the following is associated most with a ripe banana?”
I forgot those exist and interpreted it as “Would you sacrifice performance for one of these features?”
Am I stupid?
That’s how I treated it too. I took it at face value.
I have modern hardware so I don’t care too much about browser performance. All browsers perform well on my hardware. Obviously some are more lightweight and optimised, but I have no doubts about my ability to comfortably browse the web on my hardware, so all the performance questions I tended to rank in the middle (ie not most or least important) as I don’t tend to notice browser performance.
I also thought it was a feature vs performance question. How can it be used as a control question?
Things I want from Firefox/Mozilla, in no particular order:
- Just hire the uBlock Origin guy, Chrome doesn’t want him
- Dissolve the Mozilla Corporation, start a Patreon or whatever
- Foxkeh plushie
I am willing to compromise on the “unreasonable” ones 🦊
You folks are really exaggerating. How is this survey weird? The random questions in groups of 3 make it easy to compare 3 features instead of rating 60 different features by most wanted to least wanted. In aggregate from thousands of replies, they can sort all answers.
It’s the “most wanted” language. I don’t blame common folks associating “most wanted” with “I want this!” when in fact they don’t mean it.
But the language is clear. From these 3 features, choose the one you want the most and the one you want the least 🤔
The language is not clear.
" As a vegetarian, which of these three options you want the least and which you want the most? You MUST choose to continue:
- Chicken sandwich.
- A kick in the balls.
- Pork sandwich.
"
You are saying that a reasonable person (vegetarian in this case, and disclaimer: I am not vegetarian) would say “well, I want a kick in the balls the least, so I’ll choose that. Now, fuck, I HATE chicken sandwiches and I HATE pork sandwiches. They both make me puke. But if I have to choose, I guess I’ll go for the chicken sandwich. Hey pollster, I want the chicken sandwich the most.” And the pollster writes “Chester wants the chicken sandwich the most.” Yeah, very clear.
That’s already suggestive. What if you want none of them, and strongly so?
