residential_London_comments
'comments.csv' is a dataset of 30,393 public comments left in response to planning applications for new residential developments in London. The dataset includes comments made in response to planning applications filed between 1/1/2021 and 1/1/2025. It covers eight of Londons local au
‘comments.csv’ is a dataset of 30,393 public comments left in response to planning applications for new residential developments in London.
The dataset includes comments made in response to planning applications filed between 1/1/2021 and 1/1/2025. It covers eight of Londons local authority districts: City of London, Westminster, Lambeth, Southwark, Newham, Barnet, Brent and Ealing.
The dataset contains the following columns:
| Name | dtype | Description |
| id | int | id generated during scraping |
| council | string | The local authority district (LAD) of the planning application |
| comment_id | string | The comment ID used by the LAD |
| application_id | string | The planning application ID used by the LAD (sometimes referred to as planning reference) |
| address | string | Comment address |
| stance | string | Registered stance to planning applciation (‘objects’, ‘support’, ‘neutral’) |
| data | datetime | The date and time the comment was left online |
| comment_text | string | Freetext of the comment |
| add_date | datetime | The date and time the comment was webscraped |
| lat | float | Latitude extracted from the comment address using Nominatum |
| lon | float | Longitude extracted from the comment address using Nominatum |
| cleaned_comment_text | string | Freetext of the comment with proper nouns (names and addresses) removed |
| lsoa_code | string | The LSOA code of the comment |
| sentiment_score | float | The sentiment score of the free text |
The dataset is complete, apart from the addresses (‘address’, ‘lat’, ‘lon’, ‘lsoa_code’) which are missing for 20% of the entries. This is due to the council websites not recording the addresses of all comments.
The column ‘application_id’ linked this dataset to the Planning London Datahub – the GLA dataset of all planning applications filed in London.
This dataset is used in our analysis of NIMBY responses to planning applications – you can find out more through the GitHub repo: Bea-Taylor/nimby_nlp.
The additional datasets of ‘comments_with_topics.csv’ and ‘topic_names.csv’ record the topics generated using BERTopic.
📤 Share this page
Found this useful? Share it with your network.
Files are hosted on the source repository. Click download to access the full dataset.