10

Stack Exchange recently started a collaboration project with the University of Melbourne, in an attempt to improve the automatic detection of duplicate questions.

More information on the project and a call for help from the community can be found here: Project Reduplication of Deduplication Has Begun!

In it, Tim Post mentions that Doris Hoogeveen (me) will post a message on the different meta sites to let you know that we're ready for help. This is it!

As a brief summary, the project is about finding ways to automatically determine if two questions are duplicates, and also to detect the opposite: falsely tagged duplicates. To validate our methods, we need manual judgements from people who really know the data. This is where we need your help. It is impossible to manually annotate all the data from Stack Exchange, and so we have applied several filters to make sure we only show you questions pairs with a high likelihood of being duplicates.

Participation is completely optional, and there's no minimum time requirement. There's a reputation requirement however: only people who have duplicate question tagging rights on the site can participate. That is people with at least 3000 reputation points, or a gold badge. This is to ensure the annotations are compatible with StackExchange's quality standard.

If you are interested in participating in the project, please head over to the annotation interface, which can be found here: http://hum.csse.unimelb.edu.au/se-annotate/

You can log in with your Stack Exchange account, so no sign up is required. The site also contains some more information on the project, which might answer some questions you may have. And I'm also happy to answer them here.

Thank you!

7
  • 2
    Interesting project. Happy to help.
    – Fezter
    Commented Nov 11, 2016 at 1:42
  • 1
    I tried a couple of questions (will try more later) and found it very difficult to flag duplicates. Frequently, it was comparing different subjects being each one linked to different software/platform. Probably this was made intentionally; anyway I hope you can find a solid signal on your algorithm. Thanks.
    – Andre Silva
    Commented Nov 12, 2016 at 23:59
  • 2
    Hi Andre, the idea is definitely to build a system that does a better job than the pairs you are shown. Many of those won't be duplicates. What you're seeing is not the output of a fine-tuned system, but a selection of pairs we would like to get labels for, so we can use them to build such a system. So don't be disheartened by pairs that are not duplicates. They still provide us with valuable information.
    – Monozygotic
    Commented Nov 13, 2016 at 0:18
  • I don't agree that simply having a gold badge should be a qualification for participation. Not to point anyone out but please see the qualifications of this user for example: gis.stackexchange.com/users/26142/user26142 ... Strunk and White or Deputy are probably better badge qualifiers that show a decent level of understand of the posts.
    – kttii
    Commented Nov 15, 2016 at 18:16
  • 1
    @kttii, I see what you mean, but I'm not the one that has decided on the system of privileges. I'm simply following StackExchange's system, so that the annotations I get are as compatible with the existing labels as possible, and also so that SE can potentially use them on their site(s). Your suggestion is relevant outside of this project though, so you could maybe start a new thread about it?
    – Monozygotic
    Commented Nov 15, 2016 at 22:17
  • Understandable, I appreciate the response!
    – kttii
    Commented Nov 16, 2016 at 14:04
  • @kttii I see your point too - if you decide to start a Meta Q&A about it make sure that you do that on Meta Stack Exchange rather than here at Geographic Information Systems Meta because it is broader than GIS SE
    – PolyGeo Mod
    Commented Nov 17, 2016 at 6:34

2 Answers 2

Reset to default
4

This is certainly not your fault, but I cannot help wondering how much easier your project and our task would be if:

  1. all of our question bodies had only a single question (indicated by the only question mark in the question);
  2. all of our question titles were concise summaries of their bodies; and
  3. all of our questions were tagged consistently

From what I can see many of the hardest duplicates for me to assess, from those provided and every day, emanate from the three year period prior to our Meta Q&A on Framing (asking) good Questions for GIS Stack Exchange? which is linked to from our help/on-topic page.

I think preventing a proliferation of duplicates is one of the most challenging issues we face, so I wish you well with your important project.

1
  • 1
    that would certainly make duplicate identification easier, both by humans and by computers!
    – Monozygotic
    Commented Nov 18, 2016 at 2:22
2

The project is drawing to a close, and I am planning to take down the annotation interface this weekend.

Thank you very much to everyone who helped out, and thank you for all the valuable feedback and suggestions. Hopefully the collected annotations will lead to the development of better duplicate question detection systems, especially for Stack Exchange.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .