Turning data into choropleth maps with Python and Folium #604
Comments
|
Hello @nabsiddiqui and @adamlporter, You can find the key files here:
You can review a preview of the lesson here: While processing this new lesson, I encountered a couple of queries which I’d like to ask for your help with:
Could you please confirm whether these are needed for the lesson? If so, we can host them too.
Thank you! ✨ |
|
Thank you for setting this up, @charlottejmc! -- Hello @adamlporter, Thank you for your patience and collaboration. I apologise for my confusion about how the Colab notebook you created intersects with your lesson. As I mentioned in my email, we are relatively new to handling codebooks within our lessons. The key objective of the guideline notes I shared with you is to make sure learning actions are accessible to all (whatever development environment they choose to work in) and to make sure we are able to sustain the lesson (performing technical maintenance if it is needed) in the future. What I hadn't understood upon first quick-reading, but what I understand better now is that this lesson is specifically is Colab-based. For that reason, I'd like to suggest that we foreground this fact -- I'd be keen to hear @nabsiddiqui and @hawc2's thoughts, but I might suggest including Colab in the title of this lesson. The general approach we've developed encourages authors to think of codebooks as a supporting 'asset' that aggregates all the code used in the lesson, allowing readers to run it directly in the cloud, no matter their local technical specifications (as you very neatly express in the lesson). In this case (of a lesson specifically centred on using Colab as a workspace for creating the chloropleth maps) I wonder if the accompanying codebook could be pared back completely so that it contains only the code (+ essential line comments) and does not replicate any of the lesson text. We suggest that headings and subheadings are kept in place and advise that this mirrors the lesson's heading structure to support readers' navigation. We've made a new copy of the codebook which is hosted within our Google workspace. (I'll send you an invitation to edit that copy now). If we re-centre the lesson as read on our website (Markdown + inline code) in this way, some small copyediting adjustments will be needed throughout. For example, 'in the next cell' would be replaced with 'in the code block below'. Charlotte and I would be happy to help with this. Alongside this, some small changes would also be necessary in the section titled One final idea which you may have thought about previously (and which I'm sure you will discuss with Nabeel as you develop the lesson together) is how Colab enables you to show the maps as interactive visualisation. If that is a feature you'd like to explain within the lesson, you could consider creating some -- A couple of other very small notes about the adjustments we made to the Markdown file during set-up:
|
|
Thank you for setting this up. This looks great.
On Fri, May 10, 2024 at 12:39 PM charlottejmc ***@***.***> wrote:
Hello @nabsiddiqui <https://github.com/nabsiddiqui> and @adamlporter
<https://github.com/adamlporter>,
You can find the key files here:
- .md:
/en/drafts/originals/data-into-choropleth-maps-with-python-and-folium.md
<https://github.com/programminghistorian/ph-submissions/blob/gh-pages/en/drafts/originals/data-into-choropleth-maps-with-python-and-folium.md>
- images: /images/data-into-choropleth-maps-with-python-and-folium
<https://github.com/programminghistorian/ph-submissions/tree/gh-pages/images/data-into-choropleth-maps-with-python-and-folium>
- assets: /assets/data-into-choropleth-maps-with-python-and-folium
<https://github.com/programminghistorian/ph-submissions/tree/gh-pages/assets/data-into-choropleth-maps-with-python-and-folium>
You can review a preview of the lesson here:
-
http://programminghistorian.github.io/ph-submissions/en/drafts/originals/lesson-slug
------------------------------
While processing this new lesson, I encountered a couple of queries which
I’d like to ask for your help with:
-
In this assets folder, we'd also like to host any other assets that
are necessary to run this lesson. I've identified two files which are
likely to need to be hosted there: fatal-police-shootings-data.csv
<https://raw.githubusercontent.com/washingtonpost/data-police-shootings/master/v2/fatal-police-shootings-data.csv>
and
https://www2.census.gov/geo/tiger/GENZ2021/shp/cb_2021_us_county_5m.zip.
Are there any more you can think of? Hosting them ourselves ensures the
sustainability and accessibility of the lesson.
I think this makes a lot of sense. While I think both the Washington
Post's database and the Census data will probably be maintained, the links
could change (a constant challenge to referencing materials on the web), so
hosting the data with the lesson seems prudent.
-
There are a number of files already present in the notebook you have
provided us, under sample_data:
image.png (view on web)
<https://github.com/programminghistorian/ph-submissions/assets/143802849/c04303fa-da1e-4bd2-8264-e43bdf3bf404>
Could you please confirm whether these are needed for the lesson? If so,
we can host them too.
No, we don't need these. Anytime Colab starts a session, it creates a set
of "sample data" that users can play with. These files are not used for the
lesson.
- I'm seeing two images with extremely long (and slightly obscure)
links at lines 1275 and 1279 on the markdown file, which weren't included
in the package you provided by email. Would you like to keep them in? I can
save them locally, then upload and embed them using our liquid syntax
instead. However, we are committed to principles of minimal compute, which
includes ensuring our pages remain as light as possible. Since these images
are not crucial to understanding the lesson (and illustrate a rather easy
step!), I wonder whether you would mind if we simply removed them
altogether.
This woudl be fine with me. I created them for the sake of completeness; I
agree that they are an easy step that shouldn't require documentation.
Thanks,
Adam
…--
Adam Porter
- - - - -
...for if the athletes were to acquire twice the strength they possess, no
advantage would accrue to other men; but if one man were to conceive a wise
thought all would reap the enjoyment of his understanding who were willing
to share in it. -- Isocrates, Panegyricus (ca. 380 BCE), J.A. Freese,
trans.
|
|
On Fri, May 10, 2024 at 1:28 PM Anisa Hawes ***@***.***> wrote:
Thank you for setting this up, @charlottejmc
<https://github.com/charlottejmc>!
--
Hello @adamlporter <https://github.com/adamlporter>,
Thank you for your patience and collaboration. I apologise for my
confusion about how the Colab notebook you created intersects with your
lesson. As I mentioned in my email, we are relatively new to handling
codebooks within our lessons. The key objective of the guideline notes I
shared with you is to make sure learning actions are accessible to all
(whatever development environment they choose to work in) and to make sure
we are able to sustain the lesson (performing technical maintenance if it
is needed in the future). *What I hadn't understood upon first
quick-reading, but what I understand better now is that this lesson is
specifically is Colab-based*. For that reason, I'd like to suggest that
we foreground this fact -- I'd be keen to hear @nabsiddiqui
<https://github.com/nabsiddiqui> and @hawc2 <https://github.com/hawc2>'s
thoughts, but I might suggest including Colab in the title of this lesson.
This would be fine with me.
The general approach we've developed encourages authors to think of
codebooks as a supporting 'asset' that aggregates all the code used in the
lesson, allowing readers to run it directly in the cloud, no matter their
local technical specifications (as you very neatly express in the lesson).
In this case (of a lesson specifically centred on using Colab as a
workspace for creating the chloropleth maps) I wonder if the accompanying
codebook could be pared back completely so that it contains only the code
(+ essential line comments) and does not replicate any of the lesson text.
We suggest that headings and subheadings are kept in place and advise that
this mirrors the lesson's heading structure to support readers' navigation.
We've made a new copy of the codebook which is hosted within our Google
workspace. (I'll send you an invitation to edit that copy now).
If we re-centre the lesson (Markdown + inline code) in this way, some
small copyediting adjustments will be needed throughout. For example, 'in
the next cell' would be replaced with 'in the code block below'. Charlotte
and I would be happy to help with this.
Alongside this, some small changes would also be necessary in the section
titled Colab to clarify how you intend readers use the Colab notebook to
facilitate the learning actions taught by the lesson. Something like: 'We
have set up a Google Colab notebook for you to use as you work through this
lesson [...]' at that point in the text, we'll share a link to the codebook
(hosted on our organisational workspace).
This sounds like a good way to organize the material and I look forward to
working on it. Unfortunately, I am leading a trip of students to Greece for
about 12 days -- leaving on Tuesday and returning on Sunday, the 26th -- so
I won't be able to do any work on this project for the next couple of
weeks. But when I return, I would be happy to re-work this to align the
notebook / lesson in this manner.
------------------------------
One final idea which you may have thought about previously (and which I'm
sure you will discuss with Nabeel as you develop the lesson together) is
how Colab enables you to show the maps as interactive visualisation. If
that is a feature you'd like to explain within the lesson, you could
consider creating some .gif animations that express the interactivity. We
can handle .gif files in exactly the same we handle static images within
our liquid syntax. So if you want to replace one or two of the static
images with animations, we'd be happy to swap the files for you.
I don't have any experience doing this, but I would be happy to learn how
to create .gif animations to help illustrate this material.
--
A couple of other very small notes about the adjustments we made to the
Markdown file during set-up:
- I transformed the html tables you've created into Markdown tables
according to our conventions: ad51031
<ad51031>
I've checked each one, but I'd like to ask you to double-check that they
render as you intended. We can adjust as needed.
- We've added placeholder alt_text + captions for each of the figure
images, which we can work together to populate as the images are finalised.
As noted above, I will be happy to work on this when I return from the
study-abroad trip I am leading.
Many thanks,
Adam
… —
Reply to this email directly, view it on GitHub
<#604 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIBTLUSCDSRL46BXYSKLXXTZBUGV5AVCNFSM6AAAAABE52G6C2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBVGA4DCNJUHE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Adam Porter
- - - - -
...for if the athletes were to acquire twice the strength they possess, no
advantage would accrue to other men; but if one man were to conceive a wise
thought all would reap the enjoyment of his understanding who were willing
to share in it. -- Isocrates, Panegyricus (ca. 380 BCE), J.A. Freese,
trans.
|
|
Thanks very much @adamlporter, I've uploaded the additional assets for this lesson here, including the Fatal Force documentation (
We've also removed the two screenshots that illustrated downloading the files. Thanks again, Charlotte ✨ |
|
Thank you, @adamlporter (and @charlottejmc!) Hello @nabsiddiqui, You'll note that a few general thoughts and queries were raised during the Submission set-up, to which Adam has responded. Charlotte has noted a few practical details which we will be happy to collaborate on with Adam during Phase 3 Revision 1. This Submission is ready for your Initial Edit! ✨ What's happening now?Hello @adamlporter. Your lesson has been moved to the next phase of our workflow which is Phase 2: Initial Edit. In this Phase, your editor Nabeel @nabsiddiqui will read your lesson, and provide some initial feedback. Nabeel will post feedback and suggestions as a comment in this Issue, so that you can revise your draft in the following Phase 3: Revision 1. %%{init: { 'logLevel': 'debug', 'theme': 'dark', 'themeVariables': {
'cScale0': '#444444', 'cScaleLabel0': '#ffffff',
'cScale1': '#882b4f', 'cScaleLabel1': '#ffffff',
'cScale2': '#444444', 'cScaleLabel2': '#ffffff'
} } }%%
timeline
Section Phase 1 <br> Submission
Who worked on this? : Publishing Assistant (@charlottejmc)
All Phase 1 tasks completed? : Yes
Section Phase 2 <br> Initial Edit
Who's working on this? : Editor (@nabsiddiqui)
Expected completion date? : June 12
Section Phase 3 <br> Revision 1
Who's responsible? : Author (@adamlporter)
Expected timeframe? : ~30 days after feedback is received
|
|
Thank you, @adamlporter, @charlottejmc, @anisa-hawes. I plan to get this done sometime in the next week or two. I will let you know if I need anything in the mean time. |
|
Hello @adamlporter, Thank you for this wonderful article. My initial edits and thoughts are below: General CommentsComments are based on the paragraph numbers on the draft, which can be seen here: https://programminghistorian.github.io/ph-submissions/en/drafts/originals/data-into-choropleth-maps-with-python-and-folium. Overall, I really enjoyed the article and think it is a good explanation of Folium. I felt the organization was well done. I've divided my comments into these general comments and line edits.
Line EditsIntroduction Paragraph 1-The link to the Covid-19 infection/death rates leads to a page that requires you to login into ArcGIS; Combine with paragraph 2 Paragraph 3-The last sentence isn’t necessary here as it is explained in the rest of the tutorial Paragraph 4-“How to” -> “Know how to”; “One issue” -> “Understand that a key issue” Paragraph 6: Combine sentences such as: “To create the maps, we will use Folium, a Python library…” Folium Paragraph 8-"CSS and Javascript,” -> “CSS and Javascript — if you need some help, see Programming Historian’s article…. Paragraph 10-“advanced features:” -> “advanced features, such as…” Colab Paragraph 19-Paragraph is not needed as Paragraph 18 and 22 covers all this Paragraph 24-“In the next cell” -> “In the first cell”; When I ran the code in Google Colab, it says that the requirements are already met. You may want to double check if its still the case that Colab requires this to be installed. Otherwise, you can remove this and the earlier wording about geopandas. Fatal Force Data Paragraph 29-Let people know that the sample data they get will likely be different than the one displayed in the lesson County Geometry Data Paragraph 47-The first sentence is cnofusing. Match what data in ff_df? Be specific.; They are combined in the” -> “The numbers are combined in the” Preparing the Data Paragraph 56-“Doesn’t have a” -> “Doesn’t have an” Define the Question Paragraph 67-“Now that we have (1) a DF with data….->”Now that we have a DF with data (ff_df) and a DF with county geometries (counties) that share a common field (FIPS), we are ready to draw a map. Paragraph 71-“someone being killed by a police officer” -> '”a police officer killing someone” Draw the Map Paragaph 76-This paragraph is not needed. I feel the fact you are using Folium is already implied in the previous paragraph Paragraph 78-Might want to put another sentence here that says you will explain the code more below and that the reader should just paste the code for now; The code that displays the map at the end is easy to miss. It may be easier to break it into its own code chunk. Paragraph 80-“choroplet” -> “choropleth” Paragraph 81-I would recommend getting rid of the note and the following code and bullet point. You already mentioned earlier that it is not necessary to have the columns be the same name. The Problem of Uneven Distribution of Data Paragraph 83-Remove second sentence as you mention it again in paragraph 86 Paragraph 96-Dollar signs are present at the end of this. I think this is to show LaTeX? I don’t believe this is rendering properly. Paragraph 98-At the end of the paragraph, write the following “We will look at two different solutions: The Solution 1: Fisher-Jenks algorithm Paragraph 100-May need to get rid of the GeoPandas part if it turns out you don’t actually need this. Paragraph 101-Again the “m” at the end is easy to miss. Make it a seperate code block Paragraph 104-“but especially at the lower end of the scale…” -> “but the lower end of the scale is illegible.” Paragraph 105-Delete this paragraph How to add a scale-value column Paragraph 109-“For this explanation, I will assume” -> “For this explanation, I will…” Method 1: Use a Log Scale Paragraph 126-“We need not” -> “We do not need” Paragraph 133-For this, you can just write something to the effect of “For the purposes of this tutorial and its learning goals, you do not need to know the specifics of the following code. It simply replaces log values with original values.”; I would also remove the code until later since it seems that someone should paste it in before the part it is used in paragraph 135 Method 2: Cap the Scale Manually I don’t believe that you need this section. It is too confusing with all the moving pieces. Using the first method is sufficient, so I would remove this and fix the wording earlier to reflect that you are just using the log-method. Adding a Floating Information Box Paragraph 162-Combine with following paragraph Paragraph 169-Replace with the following: “Let’s see how this code functions. First, we iterate over all rows in the ‘features’ part of the GeoJSON data. Paragraph 183-See if this still makes sense if using only the log method. Add a Title Paragraph 189-“Here’s what the next cell does:” -> “Let’s look at how the code works before using it for our map.” Normalizing Population Data Paragraph 209: "Pandas allows me" -> "Pandas allows us" |
|
Hello Adam @adamlporter, What's happening now?Your lesson has been moved to the next phase of our workflow which is Phase 3: Revision 1. This phase is an opportunity for you to revise your draft in response to @nabsiddiqui's initial feedback. I've sent you an invitation to join us as an Outside Collaborator here on GitHub. This will give you the 'write access' you'll need to edit your lesson directly. We ask authors to work on their own files with direct commits: we prefer you don't fork our repo, or use the Pull Request system to edit in ph-submissions. You can make direct commits to your file here: /en/drafts/originals/data-into-choropleth-maps-with-python-and-folium.md. Charlotte and I can help if you encounter any practical problems or have questions. When you and Nabeel are both happy with the revised draft, we will move forward to Phase 4: Open Peer Review. %%{init: { 'logLevel': 'debug', 'theme': 'dark', 'themeVariables': {
'cScale0': '#444444', 'cScaleLabel0': '#ffffff',
'cScale1': '#882b4f', 'cScaleLabel1': '#ffffff',
'cScale2': '#444444', 'cScaleLabel2': '#ffffff'
} } }%%
timeline
Section Phase 2 <br> Initial Edit
Who worked on this? : Editor (@nabsiddiqui)
All Phase 2 tasks completed? : Yes
Section Phase 3 <br> Revision 1
Who's working on this? : Author (@adamlporter)
Expected completion date? : June 27
Section Phase 4 <br> Open Peer Review
Who's responsible? : Reviewers (TBC)
Expected timeframe? : ~60 days after request is accepted
|
|
@nabsiddiqui, thanks for your many good suggestions and corrections. I have edited the document to reflect them:
I've pushed the edited version of the document to Github, so everyone can review it. @charlottejmc, unfortunately these changes mean that the map images for this section (and some subsequent sections) may need to be re-generated and inserted into the article in the appropriate places. I'm happy to make the images / screen shots. Because the maps that are generated are interactive, @charlottejmc made the excellent suggestion that I create some animated GIFs that show how the maps can be scrolled / zoomed over. This would be especially nice for the info-tip box feature. I've done a bit of Googling and think I can create several of these. Where should I send them or how would I set them up in the article? Also, I realized there is a second census dept datafile that should probably be copied into the @anisa-hawes suggested that I set up a colab / jupyter notebook with just the code cells that could be accessed / run by users. I love this idea -- it could be run on Binder, so folks could execute it very easily without needing to download it to their own computers or to open it Colab. Do you have suggestions about the sort of language I should use to help users follow from one file to the other? When I've finished this, to whom should I send it? Thanks! Adam |
|
I've created three GIFs to show how users can move around the map and zoom in. I've also created a Colab / Jupyter notebook that has all the code for the article. All this material can be found in this Google Drive folder |
|
Dear @adamlporter, Many thanks for your work on the revisions in response to @nabsiddiqui's feedback. At the moment, we are providing codebooks rendered with nbviewer (for example: understanding-creating-word-embeddings.ipynb), which are openable in Colab for those readers who wish to work there, while remaining accessible for those who want to work in other cloud-based environments or locally instead. MyBinder is an interesting suggestion, and one which has also been recommended to me by someone else in our network. Thank you - I'll investigate our options. We'll download your additional images and gifs, and adjust the figure sequence as required. We'll write you a note here to confirm when the preview is updated. Thank you for your patience. Best, |
|
Hello @adamlporter, I've added the additional datafile co-est2019-alldata.csv to the lesson's assets folder, alongside the Colab notebook you've provided – and I added a short line in the lesson to point readers to it. I've also reordered all the images in the markdown file to prepare to re-upload the updated set of
Meanwhile, would you please be able to provide a caption and alt-text description for each image? You could edit the markdown file directly and insert them within the liquid syntax: Thank you very much for your patience! |
|
@adamlporter (and @nabsiddiqui): Great lesson. I only have one thought that's even remotely critical. There's a lot of techniques that are covered in the lesson beyond using Folium to generate maps. Half of the lesson is about transforming and joining the data, which in most cases is probably more work than generating the map. I mention this because I do find myself wondering how easily someone who was relatively new to mapping who read this lesson could begin to produce choropleth maps using different data that they were interested in. That data would almost certainly require different kinds of manipulations and transformations that what's covered here, and I can easily envision that person getting stuck very quickly. Having said that, that's the nature of the work. The lesson provides an example of manipulating a dataset and joining it with spatial data, and that's probably as much as it can do and people have to spend the hours learning through further research and trial and error how to manipulate their own data. As you say, the "lesson assumes some proficiency with Python." Just a thought that I offer in no way to detract from what's a very solid and effective lesson. I do have a few suggestions, which I think are all quite minor.
Stylistic¶ 4: drop the "Know how to"s. Readers will be able to "Associate latitude/longitude ..." and "Normalize data ..." Typos¶ 2 spacial |
|
Thank you @rnelson2. This is a wonderful review. @adamlporter you can make some of the copyedit changes now. I would wait until @fmvaldezg responds for more substantial edits. |
|
@rnelson2 thank you for your editorial suggestions and more for your
general observation about the arrangement of the article.
Would it make sense to take the finalized DF at the start of the lesson and
show how to turn it into a map? That way, folks would see from the get-go
how it works and could see the benefit of the process.
Then the second part of the lesson could be a "how to create a dataframe
from multiple sources" walk-through, which would show how to deal with some
of the complexity of getting to the finalized DF.
For folks who had the good fortune to find a dataset that had all the
information they needed to create a map, the first part of the lesson would
suffice. For everyone else, they would understand the goal / benefit before
wading into the morass of getting the data set up correctly.
Thanks,
Adam
…On Fri, Sep 6, 2024 at 2:39 PM rnelson2 ***@***.***> wrote:
@adamlporter <https://github.com/adamlporter> (and @nabsiddiqui
<https://github.com/nabsiddiqui>):
Great lesson.
I only have one thought that's even remotely critical. There's a lot of
techniques that are covered in the lesson beyond using Folium to generate
maps. Half of the lesson is about transforming and joining the data, which
in most cases is probably more work than generating the map. I mention this
because I do find myself wondering how easily someone who was relatively
new to mapping who read this lesson could begin to produce choropleth maps
using different data that they were interested in. That data would almost
certainly require different kinds of manipulations and transformations that
what's covered here, and I can easily envision that person getting stuck
very quickly.
Having said that, that's the nature of the work. The lesson provides an
example of manipulating a dataset and joining it with spatial data, and
that's probably as much as it can do and people have to spend the hours
learning through further research and trial and error how to manipulate
their own data. As you say, the "lesson assumes some proficiency with
Python." Just a thought that I offer in no way to detract from what's a
very solid and effective lesson.
I do have a few suggestions, which I think are all quite minor.
- Around paragraph 15, you might suggest that users open up two
browser windows side by side or use a browser with split screen
functionality to have the lesson and the Colab notebook open side-by-side
as they're proceeding through the lesson. That's what I did, and presuming
they have enough screen space on their monitor, I think it's a much more
effective way of moving through the lesson than switching back and forth.
- The code on paragraph 33 isn't in the notebook.
- In paragraph 55, you might provide a brief explanation of what a CRS
is, particularly as you use the acronym several times in the subsequent
paragraph. It's obviously an important concept for anyone doing anything
with spatial data.
- In paragraph 68, you might explain a bit more about why
.reset_index() convert map_df to a DF. It wasn't obvious to me at
least, which might just be me as I don't do a lot of programming in Python
or with pandas.
- In paragraph 81, the parameters are in a different order than they
were in paragraph 77. The ones in 77 are better for your explanation with
data appearing before key_on. That way you can swap the explanations
of those parameters so that you can mention the data (map_df) before
referencing it when describing key_on.
- I think something's missing from the first sentence of the last
paragraph: "In short, choropleth maps may be a way of displaying data and
informing readers about topics": "powerful way", "effective way"—something
like that. Even with that, the "may be" is an unnecessary qualification at
the end of the lesson. I'd suggest revising it to say something more direct
about what choropleth maps do particularly effectively.
Stylistic
¶ 4: drop the "Know how to"s. Readers will be able to "Associate
latitude/longitude ..." and "Normalize data ..."
¶ 73: You might change "The following code block initializes a map object"
to "The following code defines a python function that initializes a map
object." Very minor, obviously, but more precise.
¶ 122: I'd change var newvalue = Math.pow(10.0,
value).toFixed(1).toString() to var newvalue = Math.round(Math.pow(10.0,
value)).toString(). I found the scale with its decimal places a little
weird and maybe even off-putting as these are lives lost that shouldn't be
counted in fractions.
Typos
¶ 2 spacial
¶ 6 June, — drop the comma
¶ 10 unfamilar
¶ 43 came (instead of same).
¶ 71 passs
¶ 78 intersted
¶ 97 uneavenly
¶ 116 logrithm
¶ 119 ned
¶ 162 becuse
¶ 170 missing period at the end of the paragraph
¶ 194 I don't believe lead levels is hyphenated
—
Reply to this email directly, view it on GitHub
<#604 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIBTLUX4CZJ52YNX6AT5F33ZVIAGPAVCNFSM6AAAAABE52G6C2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZUG4YDAMZTGU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Adam Porter
- - - - -
...for if the athletes were to acquire twice the strength they possess, no
advantage would accrue to other men; but if one man were to conceive a wise
thought all would reap the enjoyment of his understanding who were willing
to share in it. -- Isocrates, Panegyricus (ca. 380 BCE), J.A. Freese,
trans.
|
|
Hello Adam @adamlporter, It's great that you're already starting to think-through your next steps, considering the first reviewer's feedback! When both reviews are in, Nabeel @nabsiddiqui will summarise them to clarify priorities for Phase 5 revisions. Thank you for your patience. |
|
Hi @adamlporter, Great lesson. I followed the steps both in the Colab notebook and by copying and pasting the code into a jupyter notebook. I generally agree with @rnelson2 that most of the lesson focuses on transforming the data. It would be interesting to present those using the lesson with the option of jumping directly to generating the map with the previously transformed data or replaying the entire lesson. ¶ 14. Add some examples of shapely data formats. ¶ 15. State the need to have a google account to execute code in colab. ¶ 16. Typo on ‘modern’ web-browser. ¶ 43. Typo on ‘same’ column names. ¶ 49. Stating why the use of s join instead of merge or join is basic. Or, paragraphs 55-57 could be at the very beginning of this section so that all paragraphs related to the sjoin process are together. I suggest referencing to the University Consortium GIS Body of Knowledge entry on ‘Spatial Joins’ ¶ 55. Include a brief explanation of the importance of defining a CRS. ¶ 63. I suggest renaming the title of this section to something like ‘Summarizing data by county’. ¶ 70. Format ‘folium.Map’ as code snipet. ¶ 130. The file located in github did not worked. The reading process is successful if the census file is used Additional coments I see a problem in the fact of using civilian deaths for mutliple years (2015-2024) and normalize it using population data for 2022 only. In my opinion, the rate should be calculated for one year only. When trying to reproduce the process in a jupyter notebook, when summarizing the In the Colab notebook, on line 5, I suggest changing the question to 'what percent of the records have latitude/longitude data?' since it is more aligned with the code result. |
|
Many thanks, Felipe @fmvaldezg! -- Hello @adamlporter, Now that both reviews are received, @nabsiddiqui will summarise the two reviews so that you have a clear path forwards for your Phase 5 revisions. Thank you, |
|
Thank you @felipelmc and @rnelson2 for the wonderful reviews, and thank you again, @adamlporter for the lesson. Based on the comments, I believe that the following changes should be made to the lesson, and then we can move forward with publishing it. I have divided them into two sections: Felipe's Comments and Rob's Comments. Rob CommentsIn regards to Rob's comment about the lesson focusing on transforming the data, I believe one way you can address this is to write a few small sentences or paragraphs outlining how the focus on the data transformation process that you have outlined here is to simulate the importance of how data is likely to need to be transformed in a real-world scenario. As Rob mentioned, "that's the nature of the work."
Felipe Comments
|
|
Hello Adam @adamlporter, What's happening now?Your lesson has been moved to the next phase of our workflow which is Phase 5: Revision 2. This phase is an opportunity for you to revise your draft in response to the peer reviewers' feedback. Nabeel @nabsiddiqui has summarised their suggestions, but feel free to ask questions if you are unsure. Please make revisions via direct commits to your file: /en/drafts/originals/data-into-choropleth-maps-with-python-and-folium.md. @charlottejmc and I are here to help if you encounter any difficulties. When you and Nabeel are both happy with the revised draft, the Managing Editor @hawc2 will read it through before we move forward to Phase 6: Sustainability + Accessibility. %%{init: { 'logLevel': 'debug', 'theme': 'dark', 'themeVariables': {
'cScale0': '#444444', 'cScaleLabel0': '#ffffff',
'cScale1': '#882b4f', 'cScaleLabel1': '#ffffff',
'cScale2': '#444444', 'cScaleLabel2': '#ffffff'
} } }%%
timeline
Section Phase 4 <br> Open Peer Review
Who worked on this? : Reviewers (@felipelmc + @rnelson2)
All Phase 4 tasks completed? : Yes
Section Phase 5 <br> Revision 2
Who's working on this? : Author (@adamlporter)
Expected completion date? : Nov 9
Section Phase 6 <br> Sustainability + Accessibility
Who's responsible? : Publishing Team
Expected timeframe? : 7~21 days
|
|
Hello Adam @adamlporter. How are you? I wondered how you are getting on with your Phase 5 revisions? Please let us know if there's anything we can do to support your next steps. Nabeel @nabsiddiqui has summarised the reviewers' suggestions to guide you, but is here to discuss any aspect. Meanwhile, Charlotte and I are on hand to help with any practicalities. Looking forward to collaborating with you to move this lesson through the final stages. |
|
Anisa,
My sincere apologies -- I thought the article was waiting for feedback from
a second reviewer. I must have missed your prior message. I will get this
done this week.
Sincerely,
Adam
|
|
Thank you, @adamlporter. No rush - We realise how busy things are towards the year-end. Please let us know if we can help in any way. All best for now, |
|
Anisa,
I've edited both the MD file and the ipynb files to incorporate the
feedback from the reviewers and Nabeel. I also took the opportunity to make
a number of corrections or changes they didn't highlight, but which I think
improve the lesson's readability / organization.
I synched these with the Github repo. Could you check to make sure I did
this correctly?
Also, one of the reviewers asked me to get rid of the decimal places on the
scales, so I created a new version of the screen shot (attached to this
email). Could you put this into the proper directory for the lesson?
I didn't know Charlotte's full name; could you edit the acknowledgements to
include it? Alternatively, could you give it to me and I could include it.
I wasn't sure if you wanted to have links to people's github handles; I
could add those, if desired.
Many thanks for your help with this process!
Adam
…On Wed, Dec 11, 2024 at 3:28 AM Anisa Hawes ***@***.***> wrote:
Thank you, @adamlporter <https://github.com/adamlporter>. No rush - We
realise how busy things are towards the year-end. Please let us know if we
can help in any way.
All best for now,
Anisa
—
Reply to this email directly, view it on GitHub
<#604 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIBTLUTEFRO4B2TMF5X4SA32FAAUDAVCNFSM6AAAAABE52G6C2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMZVGI4TKOBZGY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Adam Porter
- - - - -
...for if the athletes were to acquire twice the strength they possess, no
advantage would accrue to other men; but if one man were to conceive a wise
thought all would reap the enjoyment of his understanding who were willing
to share in it. -- Isocrates, Panegyricus (ca. 380 BCE), J.A. Freese,
trans.
|
|
Thank you, Adam @adamlporter. We really appreciate your work on these revisions. I can confirm that your commits have all been successful. Unfortunately, we cannot receive images within the comment thread, so I don't have the new version of your screenshot. May I ask if you could send this to Charlotte [[email protected]] as an attachment in a direct email? From there, we can process and upload it. Charlotte's full name is Charlotte Chevrie. I am very grateful to you for expressing thanks to us all in the way that you have. Positive feedback from contributors helps us to know that gradual improvements and adaptations to our workflow are bringing the difference we hoped for. Thank you! |
|
Thank you @adamlporter, I have received your new version of Figure 06 and replaced it in the lesson. Thank you also for your kind acknowledgements! |
|
These changes look good to me. I think we are ready to go to the next stage now @anisa-hawes |
|
Thank you, @nabsiddiqui. Hello Alex @hawc2, |
|
@adamlporter your lesson is looking good overall. I still intend to do a close line edit, but before I do that, I think it's important you to take a shot at revising this lesson to condense it in numerous ways. Your lesson is currently close to 10K words long, and we try to keep lessons to a max of 8K. In your case, there's alot of tables and code, and alot of visualizations, and it feels a bit cluttered. I'm curious to hear from you where you think we could cut things down a bit to focus on the essentials. For example, do we need tables with rows 8-10 long? Could we standardize all tables to be 4 rows or so? Are there tables or chunks of code that are repetitious and unnecessary? For example, paragraph 31 - is it necessary to show all output from printing out the possible data types? There a few cases where I'm not sure it's necessary for you to show output for code you're telling the person to run. They will see the output on their end. Is there a reason for all this being included? If not, let's cut it. Similarly, do we really need this many different map visualizations? Is it possible one or two could be removed/reduced? For example, on paragraph 123, you have a map, but no explanation for why this one is being shared. With the two prior maps, you said it showed an improvement. What does this one show that requires another reproduction? Similarly with paragraph 146, you could say more explicitly how this new generated map demonstrates a new level of improvements. Imagine a reader who is skimming through the lesson to get their bearings - those extra signposts can help alot. After the two maps I've noted here, it's important to point out you include another 4(!) map reproductions further down the lesson. Please consider if some of this can be condensed. If you could only reproduce 3-4 maps to demonstrate the value of this methodology and code, which ones would it be? The section, Adding an Information Box is very long and includes two maps. Why two? The following two sections on Minimap and Title additions could be combined into one subsection with one map demonstrating both improvements at once (in my opinion). Some of these minor aesthetic additions don't really teach anything special about the main methodology being taught here - chloropleth mapping - so I would downplay their presence here... It's a little overwhelming right now, so I'm trying to think about how to focus it for the reader. For context, I'm inclined to rank this lesson as Advanced difficulty - it assumes knowledge of digital mapping and programming, and focuses on specific types of mapping for research. I don't think you need to do much more work to further introduce this difficult lesson to the reader - it's more about how to make it streamlined for the reader to efficiently learn the key steps. I hope this makes sense. Please let me know if you have questions, and please feel free to take a shot at condensing this lesson down where you see fit. Once you take a shot, I can do a close read through and make further recommendations, so no worries if it is still over 8K when you send it back to me. |
|
Hey @hawc2. Sure. I'll take a look at this sometime soon. I'm almost caught up with most things so should be able to dedicate more time to this. |
|
Alex,
Thanks for your message. I'll be happy to work it over and try to cut it
down. I will review your suggestions and start combining / eliminating
material to make it shorter.
Cordially,
Adam
…On Thu, Feb 6, 2025 at 12:28 PM Alex Wermer-Colan ***@***.***> wrote:
@adamlporter <https://github.com/adamlporter> your lesson is looking good
overall. I still intend to do a close line edit, but before I do that, I
think it's important you to take a shot at revising this lesson to condense
it in numerous ways. Your lesson is currently close to 10K words long, and
we try to keep lessons to a max of 8K. In your case, there's alot of tables
and code, and alot of visualizations, and it feels a bit cluttered. I'm
curious to hear from you where you think we could cut things down a bit to
focus on the essentials.
For example, do we need tables with rows 8-10 long? Could we standardize
all tables to be 4 rows or so?
Are there tables or chunks of code that are repetitious and unnecessary?
For example, paragraph 31 - is it necessary to show all output from
printing out the possible data types? There a few cases where I'm not sure
it's necessary for you to show output for code you're telling the person to
run. They will see the output on their end. Is there a reason for all this
being included? If not, let's cut it.
Similarly, do we really need this many different map visualizations? Is it
possible one or two could be removed/reduced? For example, on paragraph
123, you have a map, but no explanation for why this one is being shared.
With the two prior maps, you said it showed an improvement. What does this
one show that requires another reproduction? Similarly with paragraph 146,
you could say more explicitly how this new generated map demonstrates a new
level of improvements. Imagine a reader who is skimming through the lesson
to get their bearings - those extra signposts can help alot.
After the two maps I've noted here, it's important to point out you
include another 4(!) map reproductions further down the lesson. Please
consider if some of this can be condensed. If you could only reproduce 3-4
maps to demonstrate the value of this methodology and code, which ones
would it be? The section, Adding an Information Box is very long and
includes two maps. Why two? The following two sections on Minimap and Title
additions could be combined into one subsection with one map demonstrating
both improvements at once (in my opinion). Some of these minor aesthetic
additions don't really teach anything special about the main methodology
being taught here - chloropleth mapping - so I would downplay their
presence here...
It's a little overwhelming right now, so I'm trying to think about how to
focus it for the reader. For context, I'm inclined to rank this lesson as
Advanced difficulty - it assumes knowledge of digital mapping and
programming, and focuses on specific types of mapping for research. I don't
think you need to do much more work to further introduce this difficult
lesson to the reader - it's more about how to make it streamlined for the
reader to efficiently learn the key steps.
I hope this makes sense. Please let me know if you have questions, and
please feel free to take a shot at condensing this lesson down where you
see fit. Once you take a shot, I can do a close read through and make
further recommendations, so no worries if it is still over 8K when you send
it back to me.
—
Reply to this email directly, view it on GitHub
<#604 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIBTLUW22TA6H7PBYMRCCL32OOSUZAVCNFSM6AAAAABE52G6C2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNBQGY4DKNJTGA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Adam Porter
- - - - -
...for if the athletes were to acquire twice the strength they possess, no
advantage would accrue to other men; but if one man were to conceive a wise
thought all would reap the enjoyment of his understanding who were willing
to share in it. -- Isocrates, Panegyricus (ca. 380 BCE), J.A. Freese,
trans.
|
|
Thank you for your patience and your energy, @adamlporter. If you decide to remove any images, please simply let me know which figure numbers you removed, and I will take care of updating the filenames and captions on those remaining. |
|
Alex,
I've edited the lesson and pushed a revised version onto the PH github repo.
Changes include:
* Removing a few rows from all the table displays and from the output of
the .head() and .sample() functions.
* eliminated material about boolean masks and simplified the description of
how to tell how many missing values of data there are.
* combined / simplified discussions in several places in the document.
* eliminated material about the mini-map and adding a title
* I think we can eliminate four maps:
fig 2 - shows basic map
fig 3 - box plot (to show data distribution problem)
fig 4 - map with fisher-jenks algo
*fig 5 - eliminate (map with basic log scale)*
fig 6 - map with improved log scale
fig 7 - map with normalized data
fig 8 - box plot
fig 9 - box plot
fig 10 - normalized data, log scale
*fig 11 -- eliminate (simple pop-up diagram example)*
fig 12 - map with complex popup
*fig 13 -- eliminate (mini map)*
*fig 14 -- eliminate (title)*
I'm not sure how to calculate word counts and I fear I've still got more
than 8K words.
If more radical surgery is needed, we could get rid of the discussion of
how to draw a floating information box. It's extremely helpful when looking
at these sorts of maps, but it would be straight-forward to eliminate.
Thanks,
Adam
…On Tue, Feb 11, 2025 at 4:21 AM charlottejmc ***@***.***> wrote:
Thank you for your patience and your energy, @adamlporter
<https://github.com/adamlporter>.
If you decide to remove any images, please simply let me know which figure
numbers you removed, and I will take care of updating the filenames and
captions on those remaining.
—
Reply to this email directly, view it on GitHub
<#604 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIBTLUTJBS6P5534AVFE55L2PHFK3AVCNFSM6AAAAABE52G6C2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNJQGM3TENZYGI>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Adam Porter
- - - - -
...for if the athletes were to acquire twice the strength they possess, no
advantage would accrue to other men; but if one man were to conceive a wise
thought all would reap the enjoyment of his understanding who were willing
to share in it. -- Isocrates, Panegyricus (ca. 380 BCE), J.A. Freese,
trans.
|
|
Hey @adamlporter, Thank you for your revisions to the lesson. The changes you've made streamline the content. I think the main issue now is target length. I would definitely get rid of any maps that you think are not needed and maybe combine the explanations of similar techniques (like the various scaling methods). Your suggestion about potentially removing the floating information box section is also worth considering, though I agree this feature is valuable for users. You can also reduce some of the inspection steps shown (like .info() and .sample() outputs by showing only the most relevant parts of the output rather than the full display. |
|
Hi @adamlporter, according to several different online word counter tools, you have between 7900 and 8300 words here. I don't think you need to completely remove anything important from your lesson. In the next phase, I will be carefully copyediting your lesson: I'll try to keep word count in mind and edit sentences down for you, as well as merge any repetitions I might find. Thank you for outlining the figures you'd like to remove. I've deleted figures 5, 11, 13 and 14, so you now have a total of 10 figures. You can check that it renders correctly in the lesson preview. I agree with @nabsiddiqui that you don't need to show the full output display if it is not essential. You can focus simply on the relevant lines (if you do so, I'd recommend adding a short note explaining this choice so that readers are aware their own output will look a little different). |
|
Hey @charlottejmc and @adamlporter. Yes, if we are ok with the length, I think this is almost ready to go, other than maybe removing the full output display when not necessary. |
|
Hi @adamlporter, it seems that your local edits created some conflicts in another lesson. We would prefer it if you worked on GitHub online, but if you must work locally, please make sure to always pull the very latest version! We're very active so the repo changes very quickly... Thank you ✨ |
|
I'm sorry! I was trying to rebase my local copy and thought I told it to overwrite all my local files with the material from GitHub. I certainly didn't mean to cause problems with other submissions. I'm very sorry! |
|
It's OK – the conflicts were easy to resolve manually. These things are often knotty and mysterious! |
Programming Historian in English has received a proposal for an original lesson, 'Turning data into choropleth maps with Python and Folium,' by @adamlporter.
I have circulated this proposal for feedback within the English team. We have considered this proposal for:
We are pleased to have invited @adamlporter to develop this Proposal into a Submission under the guidance of @nabsiddiqui.
The Submission package should include:
We ask @adamlporter to share their Submission package with our Publishing team by email, copying in the editors.
We've agreed a submission date of early April. We ask @adamlporter to contact us if they need to revise this deadline.
When the Submission package is received, our Publishing team will process the new lesson materials, and prepare a Preview of the initial draft. They will post a comment in this Issue to provide the locations of all key files, as well as a link to the Preview where contributors can read the lesson as the draft progresses.
If we have not received the Submission package by April, @nabsiddiqui will attempt to contact @adamlporter. If we do not receive any update, this Issue will be closed.
Our dedicated Ombudspersons are Ian Milligan (English), Silvia Gutiérrez De la Torre (español), Hélène Huet (français), and Luis Ferla (português) Please feel free to contact them at any time if you have concerns that you would like addressed by an impartial observer. Contacting the ombudspersons will have no impact on the outcome of any peer review.
The text was updated successfully, but these errors were encountered: