9

We have a large number of questions, such as this recent one, How to delete last part of a string starting from the 1st non numeric character?, which involve mainly basic scripting/programming knowledge, but also some knowledge of how to use the field calculator itself, and particular quirks about field calculator implementations, such as Removing return/newline (\n) character from Field using Python and Field Calculator?

Clearly field calculation is a common GIS task and there is value in these questions, but they also often attract close votes since often the only specific knowledge required is how to perform some generic programming task such as Python string manipulation, regular expressions, etc.

Should we just allow all questions or is there a line to draw where they should be closed as off-topic (general programming)?

  • 4
    +1 Very good question; it's a real challenge to answer. – whuber♦ Oct 17 '14 at 17:34
  • 1
    As a side note, the aforementioned question has now been closed as off topic, which I think is slightly unfortunate as it was a well-put question with good answers. – blah238 Oct 17 '14 at 22:31
  • 3
    Quite frankly, anything Python related has always confused me on this site. Granted, I know little enough about it, but it seems I see the same questions get asked and one gets closed and the other answered and up voted with little consistency. I know there's probably a line somewhere, but I see a big difference between general Python application and someone who is GIS needing to use it to do something in that context. It's like telling an auto mechanic wanting to hang a picture to go learn to be a carpenter (or at least ask one how to use a hammer on something other than metal). – Chris W Oct 18 '14 at 21:28
  • 4
    This is a great question. There are so many quirks to the ArcMap field calculator which require an experienced ArcMap user to debug - while other aspects of the field calculator are pure Python/VB. Coupled with the fairly-useless -9999 error messages, the field calculator is a real challenge. – Stephen Lead Oct 19 '14 at 23:32
10

Thanks for bringing this up--I have also been noticing a higher frequency of flagged field calculator questions. I believe that any question relating to a GIS field calculator is on-topic. The field calculator has its own peculiarities, such as wrapping fields in !FieldName! for ArcGIS. With this said, even the most pure Python concepts become GIS concepts, as they now have software specific requirements. I would also add that field calculator questions have been flagged more routinely within the last year or so, which suggests that the flags may be coming from one or a few individuals, not the community as a whole.

8

Excellent question!

I wonder if the solution is to look for the earliest Python Parser of the Field Calculator question and extensively edit it to become a Community Wiki Q&A entitled Using the Python Parser of the Field Calculator of ArcGIS for Desktop? that starts something along the lines of:

This is a Community Wiki Q&A which is intended to cover any aspects of using the Python Parser of the Field Calculator of ArcGIS for Desktop that are not pure Python.

Pure Python questions which are beyond the beginner examples provided as answers below should be researched/asked at Stack Overflow.

Answers could range from Removing return/newline (\n) character from Field using Python and Field Calculator?, to any of its other quirks like wrapping fields in exclamation marks (!FieldName!), and to some of the most commonly asked "beginner Python" tips that come out of all the Q&As we have seen so far.

Candidates for the "beginner questions" to try and cover off on could come from this FAQ.

The intention would be to point all future Python Parser beginner questions back to this one as duplicates, and to redirect any complex questions related to the Field Calculator that are nevertheless pure Python to be researched/asked at Stack Overflow.

A similar approach could be used to create a Community Wiki for Using Python with the Field Calculator of QGIS.

The aim of creating these Canonical Q&As is to avoid the need for the community to answer very similar questions over and over again which adds little new value and means that visitors to the site have to wade through potentially hundreds of Q&As rather than a single very well written Q&A from the community on the topic.

Possibly relevant: What does Stack Overflow Documentation mean to GIS Stack Exchange?

  • Sounds good, but I prefer question titles to start with one of the "6 Ws", e.g. "How to use ...". It's not really any longer but is clearer, more grammatically correct, and easier to read, IMHO. As for a candidate question, I might suggest this one: gis.stackexchange.com/questions/8695/… – blah238 Oct 17 '14 at 3:58
  • @blah238 I used to agree with you on starting questions that way, but after reading this Meta SE Q&A, particularly Jeff Atwood's answer, I started opting for more conciseness. That looks like a great candidate question. – PolyGeo♦ Oct 17 '14 at 4:04
  • I've seen that one before, I just disagree with that particular point, as to me it's the most natural way of asking questions, and indeed most of the FAQs are phrased this way. But, moving on -- perhaps we should consider making the question language-neutral, as VBScript is still sometimes used, but probably not enough to get its own canonical Q&A. Additionally there is the Calculate Field tool which uses a very similar syntax, but is not restricted to ArcGIS for Desktop, so we might also want to cover that. – blah238 Oct 17 '14 at 4:51
  • 1
    Another fun topic is using the Calculate Field geoprocessing tool within Python. This involves either mixing VBScript and Python, or escaping Python within Python. Not pretty, but some people choose to do it instead of using cursors for performance reasons or just because they don't know better. I found a surprising number of these via a Google search. – blah238 Oct 17 '14 at 4:57
  • 1
    @blah238 That FAQ list does make a case for your preferred titling. I agree that Field Calculator and the Calculate Field tool should be grouped but I think we should keep VBscript out of scope for the canonical Q&A - not to prevent your fun but since it is targeted at Python newbies I think any additional complication will reduce its readability and usefulness. – PolyGeo♦ Oct 17 '14 at 5:03
  • +1 Thank you for the excellent suggestions. I just want to note that there is more than one "Field Calculator": most GISes have something like it. There would need to be a CW FAQ thread for each such GIS--and, eventually, multiple such threads for ArcGIS because it is constantly changing its behavior. – whuber♦ Oct 17 '14 at 17:36
  • How do you all feel about "one field calculation FAQ to rule them all" -- i.e. a canonical, community-wiki question which has sections and answers for all the different implementations, as well as the guidelines for closure, that we can point any off-topic or duplicate questions to? I feel that would be the most beneficial and easiest to implement. We could start small and grow it as needed, without having to divide up the useful information into separate Q&As based on the language, version or vendor -- and keep in mind the "answers" could be as simple as links to already-answered questions. – blah238 Oct 17 '14 at 21:25
  • @blah238 My first thought is that I am not a fan of mammoth Q&As, because I think visitors generally come looking for just an answer or two, and hope to get what they came for by just reading a few paragraphs to think "That's what I need to know!" and not too many answers (maybe 5 max.) to think "That's my answer!". My thinking was a specific language and product (no need for version) would be more likely to "right-size" the Q&A. However, I would be keen to see you try out what you think will work best. – PolyGeo♦ Oct 17 '14 at 21:55
  • @PolyGeo I would agree if we were talking about one specific question, but I see this as more of a FAQ, primarily consisting of links to other Q&As, but also general guidelines about the topic. Answers to the FAQ could be a collection of questions and guidance pertaining to one particular implementation/product/language, so that at most there would probably be on the order of 5 answers, each maintained by the users most interested in that particular implementation. Kind of a "one-stop shop" for all your field calculation needs :) – blah238 Oct 17 '14 at 22:11
  • @PolyGeo I have added a proposal as an additional answer. Let me know what you think. – blah238 Oct 17 '14 at 23:51
4

These are one of my favorite types of question, and I'm personally disappointed when they are closed as off topic. The Python questions are never going to stop, so GIS SE might as well adapt instead of turning them away.

  • 1
    Our discussion above is about attracting people who are looking for answers to Python questions related to the GIS field calculators, and having answers to something similar to the most common of them ready to just read, adapt (to get a learning experience) and use, rather than having to ask and wait for an answer. – PolyGeo♦ Oct 27 '14 at 6:42
  • 1
    I agree with @ian, python questions are never going to stop as they are often asked by people who are learning or have basic experience (including me!). My impression is that the "easy" questions are an indication of the poster not bothering to do some research into their question. So I'm of the opinion if they frame their question in a GIS context and\or show some evidence that they RTFM then its an on-topic question. If it is just a python question it should be closed or migrated to the appropriate SE forum. – Hornbydd Oct 30 '14 at 13:04
3

Here is my proposal for a new question to be designated as the canonical field calculation FAQ. I figured I'd propose it here rather than on the main site so it can be refined and discussed. I'd also suggest creating a new question rather than repurposing an existing one so that we have full control over its contents (a blank slate, basically).

Field calculation FAQ

Note: This question is intended to be a canonical resource for all things related to the task of calculating fields in GIS software. Please follow the suggested guidelines below for responding to and maintaining this Q&A.

Summary

Calculating fields is a very common GIS task typically involving writing an expression or function in a scripting language to perform some operation on the values of a field (column) in a database table. It is similar to an SQL UPDATE statement.

Various GIS software implement graphical user interfaces (GUIs) or tools which allow the user to interactively build or manually enter an expression, function, or statement to carry out the operation.

Because there is considerable overlap in the skills and knowledge necessary to write the logic for a field calculation between our site and other sites, such as StackOverflow, field calculation questions are often closed as off-topic, or as duplicates of existing GIS SE questions. Most field calculation questions are tagged with the tag.

Therefore, this question is to be used to collect and reference information about the essentials of field calculation, moderation guidelines, and frequently-asked questions (FAQ's) here, in order to serve as a "one-stop shop" for the topic of field calculation.

Guidelines

  1. Answers to this question should focus on a particular field calculator implementation, such as the ArcGIS Field Calculator or the QGIS Field Calculator. Some implementations may use more than one programming language to input expressions (such as Python and VBScript in the case of ArcGIS). In this case, separate answers specific to each language may be created.
  2. Answers to this question should take the form of an FAQ, with a bulleted list of topics, links to existing Q&As or other reference materials, and summaries of the most essential information about a particular field calculation implementation. An example FAQ is the FAQ for Stack Exchange sites.
  3. If an answer to this question covering a particular implementation already exists, that answer should be maintained or updated with new information, rather than adding a new answer.
  4. Update the Table of Contents section below to add anchor links to specific answers and topics. Note: Figure out how to do this and add instructions :)

Table of Contents

TO DO

Thoughts/critique/suggestions?

  • I think this is worth a try. Perhaps say whether you intend including sections on "How can a question be added to this FAQ?" like the FAQ for Stack Exchange sites does at its bottom. I think keeping it less formal, at least initially, will be the way to go. – PolyGeo♦ Oct 18 '14 at 0:00
  • I think the guidelines cover that -- basically, just add an answer for a particular implementation if one does not exist, or update an existing one. The difference with the FAQ for Stack Exchange sites is that the individual questions are actually separate Q&As, whereas I'm proposing that answers (consisting primarily of lists of existing Q&As) be added to the "Field Calculation FAQ" directly. Additionally, theirs is a meta FAQ while ours is a "main" FAQ, so we do not have the same liberties with regards to faq tags. Let me know if that is still unclear. – blah238 Oct 18 '14 at 0:06
  • Perhaps include that clarification in the "template" above or maybe there is a better example FAQ that you can link to from Main SO, or another Main SE. I think the clean slate idea works better than re-purposing an old Q&A. – PolyGeo♦ Oct 18 '14 at 0:11
  • I will have to look for one later, but if anyone comes up with a better example or additional clarifications, feel free to edit my proposal. – blah238 Oct 18 '14 at 0:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .