Saferview - crime, fear and mapping

New Venn diagram for Orison

2012-01-10T23:44:00.002Z

Orison left me a note saying he was fascinated by my Venn diagrams, but I have progressed. This is the one I have included in my thesis. Almost finished.

I have looked at the statistics. I am still getting over 50 visits a day and I have not published much for almost a year. Once I have submitted my thesis I intend to start publishing again. I have a lot to write about.

Does the National Statistician understand crime data?

2011-06-08T15:53:00.004+01:00

I have broken my silence because I am seriously concerned. The National Statistician's Review of Crime Statistics in England and Wales was published yesterday. This is what my last post but one referred to. I got the person of the National Statistician wrong. It is now Jil Matheson pictured above with Prime minister David Cameron and on her own.

Unsurprising she has recommended that the Office of National Statistics should take responsibility for crime statistics away from the Home Office. This recommendation 5. Recommendation 1 is:

"The body responsible for the publication of crime statistics should seek to improve the presentation of the statistics to give users and the public a clearer understanding of the overall picture of crime, by providing the major and other sources of crime statistics together with additional contextual information."

This will only happen if the statisticians understand the raw data they present. Figure 1 shown above of the report and the explanatory paragraph, clearly shows they do not understand the scope of police incidents. At best the paragraph and figure present a confusing, unclear explanation. In fact it is totally wrong, worryingly wrong, because whoever wrote paragraph and compiled the figure does not have the first clue about the reporting mechanisms resulting in police recorded crime and the contents of the incident database.

The incident database has details of less than 50% of notifiable offences that end up being counted as police recorded crime but contains hundreds of thousands of incidents that come within recorded crime categories that are not counted as police recorded crime. Additionally not all of the 4.3 million notifiable offences are mapped because either they have no mappable location or the fall outside the categorises that are mapped.

What is Geographic Information Science (GISc)?

2011-03-14T12:35:00.003Z

I am aware that I have not posted for sometime. This is because I have been trying to get my thoughts together to start writing my PhD thesis. I thought I would post part of my introductory section discussing Science and Geographic Information Science (GISc). Without GISc there would be no crime maps so it is quite important. Everything I know about the subject and a lot more is contained in the excellent bestselling book in the world on the subject, that has recently been published in its 3rd edition - "Geographic Information Systems and Science" whose lead author just happens to be one of my supervisors - Professor Paul Longley.

What I am trying to do is explain why GISc is undoubtaby a science (something I needed to be convinced about when I first approached the subject) and to start laying the ground work to justify creating and comparing classifications between areas using crime and policing data.

Science

This section discusses the scientific context of the research. The discussion adopts a simplified approach, guided by others who have studied and written about the nature of science, its definitions and its academic divisions. These processes of definition, simplification, clustering and classification, often in an inexact and overlapping way, are of course central to the scientific method and central to the research carried out in this thesis.

The word ‘science’ is derived from the Latin via Old French and Middle English to denote ‘knowledge’ (Oxford online Dictionary). The purpose of all science is to transform raw data by analysis and/or experimentation into knowledge (Oxford online Dictionary, Longley et al 2010). Knowledge in this sense is an understanding of the system that is being studied so that it can be described by relationships, laws or theories that model its key dynamics. This is a process that goes further than describing a system by providing an insight into how it works. Part and parcel of understanding a system is to be able to identify uncertainties and where possible quantify them; the statistical use of confidence levels of results is an example of this. This quantification of variables and relationships allows research to be reviewed by peers with a view to validation and the generation of new knowledge through further scientific research.

Geographic Information Science (GISc)

The over-arching methodology used by this thesis is Geographic Information Science (GISc). Geography is defined by the (Oxford online Dictionary) as;

“the study of the physical features of the earth and its atmosphere, and of human activity as it affects and is affected by these, including the distribution of populations and resources and political and economic activities.”

The use of the word ‘Information’ refers to the organised collection of data, for instance in a database. GISc relies on Geographic Information Systems (GIS) for the storage of geographic data and organising it so that it can be used for scientific purposes. The data in GIS come from a myriad of different sources, some of which are specific to the creation of mapping the surface of the earth such as satellite imagery and others that are by-products of other data gathering activities such as census data and land-ownership. The data from the disparate data sources are linked by the fact that each row of data in every data set in the system is geo-referenced. This allows the joining of the different data by location or the keeping the data sets separate by layering the data by location or a combination of both. This leads to mentioning the special nature of spatial data that is based on the Tobler’s first law of geography;

“Everything is related to everything else, but near things are more related than distant things” (Tobler 1970).

Tobler’s law refers to spatial autocorrelation which affects the independence of variables which are analysed spatially. This spatial autocorrelation is two dimensional (unlike temporal autocorrelation which is one dimensional) because it radiates out from a fixed point. Both these facts necessitate the use of spatial statistics if the spatial nature of variables is being assessed. The complex nature of the application of Toblar’s Law to crime and policing data is discussed further in this thesis.

The creation of maps is a scientific process, representing the real world is a process of scale selection, data choice and emphasis based on purpose, generalisation and geovisualistion. These methods of representation are by their nature incomplete and imperfect as the only way to perfectly represent the real world is using a 1:1 scale which is pointless and totally impractical. This means that irregular lines on maps, such as coastlines and roads are simplified by excluding the smaller twists and turns; polygon zones are created to show features that are pertinent to the purpose of the map that generalise or amalgamate entities being represented. This can either be completely objective based on measurement and statistical criteria or purely subjective based on the eye of the mapmaker; often it is a combination, this where science meets art. Maps are designed to be intuitive (once the spatial language is understood) by duplicating how human-beings perceive and make sense of the real world both visually and intellectually. Our perception of things goes from the general to the more and more specific as the resolution increases; in a geographical sense, without the aid of maps, this involves travel, for instance moving from seeing a wood in the distance to entering the wood and only being able to see the trees. This raises the question, “Is it better to see the wood or the trees?” The answer is of course “It depends on your purpose.” The conclusion is that, in line with the saying “You cannot see the wood for the trees”, in many circumstances the general can be more important and informative than the specific.

Imperfect clustering and classification are fundamental to GISc. They can be applied to label areas on a map as forests even though there are areas within them of agricultural land or showing the residents of an area as “small town seniors” even though youth offenders are known to live there. The importance of this methodology to this thesis is that it allows hypotheses of relationships between different layers at the same locations to be proposed and comparisons between different locations to be investigated on a single or multi-layer basis. To continue with the tree example, it would be logical to have a polygon denoting timber production on a layer to spatially coincide with a polygon on a separate layer denoting forest but not necessarily all forest polygons spatially coinciding with timber production: If a polygon denoting timber production is nowhere near a forest polygon, this is worth further investigation and explanation.

GISc is a pure science because it is used to empirically measure and represent geographic characteristics and attempts to understand how they interact with each other. It is also an applied science because much of this knowledge is put to practical use, for example Internet crime maps for the public.

Crime Statistics, are the Statisticians to blame for lack of confidence?

2011-02-19T12:59:00.036Z

Sir Michael Scholar, the UK's National Statistician, pictured above, has been asked by the Home Secretary to carry out a review of crime statistics with the purpose of increasing public confidence in them. He has published a questionaire that can be found here that anyone can complete and submit. Below are my thoughts. I probably have not started off very well by putting some of the blame for low confidence in the crime statistics on the statisticians themselves, but I try to tell it as I see it. Having heard Sir Michael speak at the Royal Statistical Society I am sure he and his staff will be receptive to my views.

Q1: Responsibility for the publication of crime statistics is to be moved out of the Home Office. Who should now assume this responsibility to increase public trust in the crime statistics?

It should be the responsibility of a body independent of government but reports to the Home Office in a similar way that the Bank of England is independent of the Treasury but reports to the Chancellor of the Exchequer.

It would a mistake for it to be given to the UK Statistics Authority as I think that one of the major problems that needs to addressed is the fact that it is obvious that the statisticians do not understand the data fully, given its legal and procedural subtleties and have difficulty in producing reports that the policy makers or the public properly understand.

There is far more to crime statistics than just statistics, a major part of it is understanding crime and policing; which are the domains of criminology and police science. With the increasing use of maps to communicate the statistics to the public the discipline of Geographic Information Science is becoming more and more important.

Q2: Is there also a case for transferring responsibility for the management and/or compilation of data collected from the British Crime Survey and the police ? If so, where?

The National Audit Office should be seriously considered for police recorded crime. Even though the data is not financial per se, the skill sets of auditors are very compatible. One of the main challenges to the accuracy and integrity of police recorded crime statistics is that they are used as primary measures of police performance. The procedures for managing and collecting crime statistics to ensure uniformity within and between police forces needs to be overseen by people who have the forensic skills to identify deliberate under recording. As crime reduction and clear-up rates are directly related to individual police officers promotion prospects and pay the auditors should also have sanctions that they can enforce on individuals.

The British Crime Survey (BCS) should be totally independent of the police recorded crime statistics and pressure from the government. It has been a mistake to have one official publication trying to link police recorded crime with the BCS because it makes it look that the two sets of data can be compared directly. In most cases there are reasons why they cannot (due to business and youth crimes not being included for instance) the only notable exception is residential burglaries (but the fact that the BCS uses a rolling 12 month period and the police statistics use a fixed 12 month period affects this and the others crimes). As far more data is collected by the BCS than are published in the resulting dual publication it has the feel that the BCS data is being used to put a political spin on the police recorded crime data. This is at the hub of the need for independence. As important is the long term funding of the BCS so there is not a pressure to please the funding master/mistress to gain continued support.

The linking of different crime data sets should be done independently by academics and those putting them to practical use.

Another solution is to make the two data sets directly comparable which is preferable – see the 4th paragraph of the answer to Q5.

Q3: Currently, the Home Secretary determines what is recorded by the police as a crime and approves the Home Office Counting Rules for crime and statutory data requirements from the police. Should this continue or would public trust in the statistics be enhanced if this responsibility moved elsewhere? If so, where and why?

The Home Office should determine police priorities and ensure that bureaucracy is minimised; these are levers for doing this. The agency managing the collection of the police recorded data, my suggestion the National Audit Office, should be required to comment on whether the rules and requirements properly reflect the level and nature crime in the country.

Q4: The Terms of Reference for the review asks for consideration of the current definitions of crime. Do you have any comments?

Crimes in crime statistics always have a legal definition and sometimes in addition a descriptive label. For instance in law there is no offence of burglary dwelling, snatch, armed robbery, it is burglary, theft and robbery (with firearms offences). The labelling in the main allows offences against vehicles to be distinguished and with a bit of delving those offences against people; this should be made clearer. Equally important are those crimes that occurred in a private place or a public place. Under the present labelling it is often not possible to make any inferences in this regard. I think this is vital to understanding the crime problem and police performance.

There are ten Home Office (HO) crime types. The one that leads to the most misunderstanding, confusion and distrust is HO crime type 1 – violence against the person. It is not helped by the fact that statistics that are derived from it are referred as ‘violent crime’ by Home Office publications and politicians. The public order offence of violent disorder (that usually includes violence against the person but not necessarily) which is not included in HO crime type1, shows that violent crime in law applies to criminal damage as well as assaults. This aside, robberies, sexual assaults, aggravated burglaries, kidnapping and arson endangering life which are specifically violence against the person offences are not included in HO crime type 1. The inclusion of common assault (section 39 of the Criminal Justice Act 1988) in the sub-type of ’without injury’ is misleading because there almost always is an injury but not of a permanent nature; only assaults without battery should be included in this sub-type.

If the category of violent crime is important to the public and politicians, which I believe it rightly is, then all crimes that are violent should be included in that category with sub-categories as necessary. If this means double counting of certain crimes because they also meet the criteria of another category, this should not be a problem as long as it is properly explained.

Q5: It has been said that the crime statistics provide a partial picture. What, if any, are the main gaps in Home Office crime statistics that you feel should be addressed as a priority?

Official Crime Statistics have taken advantage of police computerisation of crime records in the collection of data but not in the scope of what data is collected.

Police accountability and crime prevention would be aided by counts of characteristics (age, gender, ethnicity, geodemographic group for instance) of victims of crime and suspected offenders broken down by crime types and offences. Advances have been made in providing location data about crimes through the medium of maps on the Internet, the accountability and crime prevention advantage could be further enhanced by the provision of data of when (date, day, hour) offences had been committed and/or discovered.

The dark figure of unreported and unrecorded crimes is very important to assessing police performance and confidence in police. I would argue the best proxy measure of confidence in police and police efficiency is the size of that dark figure which is known to vary from offence to offence. The BCS was originally set-up to shed light on this dark figure but as I have mentioned in my answer to Q2 the lack of direct comparability of the two data sets affects the confidence in the accuracy of the estimates produced. A reliable figure can be produced of unreported and reported BCS crimes but the figure for reported and unreported police recorded crime is less reliable. This makes the important figure of those crimes reported to the police but not recorded by the police also unreliable.

The solution to this is not to change the methodology of the BCS but to extract only those crimes that are covered by BCS from the police crime computers for comparison purposes. This is feasible given the data contained within each police crime record and the modern searching power of computers. If the characteristics of victims was also included then very useful assessments of police engagement with different communities would result and provide a good proxy indicator of the level of confidence different communities have in the police. The accurate assessment of the nature of the dark figure for different offences would provide a good measure of police effectiveness and efficiency. This would balance police recorded crime reduction targets that can be achieved through the public feeling that it is not worthwhile reporting crimes to the police.

The Reassurance Gap that led police to realign resources from response policing to neighbourhood policing arose because the public perceived that the crime problem was getting worse even though the official crime figures were showing year on year improvements. The analysis of this included an assessment that one of the reasons for this was that the official crime figures did not match the experiences of people. Drawing on Incivility Theory from the USA and the newly developed Signal Crime Perspective in the UK it is generally agreed that the observable nature of crime and disorder rather than officially collected crime figures influence people’s perception of crime and therefore their level of fear of crime. The all pervasive low level disorder (antisocial behaviour or incivilities) has as much influence, if not more, than the observation of serious crime on people’s perception. Police recorded crime, because of the way it is collected and legally categorised will never be totally in tune with people’s perceptions. My research includes analysing police incident data to see if the data can be used to produce maps that more closely match people’s observations of crime and disorder. The importance of this is that if policy makers and police have a view of the policing problem in a neighbourhood that is not matched by the communities living and frequenting there then there will be a mismatch of priorities and expectations. Research has shown that trust is based on good engagement and communication, crime statistics and maps based on them can either alienate or engage. Police incident data is being used to map anti-social behaviour but it should be used much more widely to reflect the nature and levels of violent crime. Police incident data has the added value of having good temporal attributes and shows details of the police response and priorities, and also the priorities and engagement of the public.

Q6: What are the most important considerations for trustworthy crime statistics?

The most important consideration for trustworthy crime statistics is that they are understood. The Home Office Counting rules, legal definitions, Home Office Crime Types, especially in relationship to violent crime are all barriers to proper understanding of police recorded crime. The intricacies of methodology and statistical techniques are barriers to properly understanding, evaluating and therefore trusting the BCS statistics.

The police incident data set is in everyday language, it reflects what people observe and it is easy to explain the origins and validity of the data. It cannot replace the police recorded crime statistics or the BCS but it enables the public to know and understand what the police in their local area are dealing with on a day to day basis. My evaluation of police incident data shows that police recorded crime probably accounts for fewer than 50% of their workload of crime and disorder related. This means that crime statistics would be considerably enhanced as a performance measure if police incident data were included.

Q7: What do you consider to be the main strengths of crime statistics?

The main strength of crime statistics especially when applied to a local level through mapping is that they are extremely popular. This provides an excellent communication and engagement tool that should be used positively and honestly.

Police need to exploit this popularity by including police activity data within their maps to reassure the public and demonstrate their worth to society that is far more than what is revealed in the present crime statistics.

Q8: Do you have any other views you wish to feed into this review?

The spatio-temporal characteristics of offences are extremely important to crime prevention as are the characteristics of victim and offenders for crime prevention and public engagement. This points to an integrated system of maps using police recorded crime, police crime system, police incident management system and BCS as source data. My PhD research involves designing such a system with spatio-temporal clustering and classification at its heart.

CAD incidents - validation of the k means classification method

2011-02-12T22:29:00.000Z

London police violent and acqusitive crime incident in 2009 numbers each day

This post is really quite interesting if you have following my blog.

I have carried out a simple analysis to test three important aspects of my research;

How good is the SPSS 17 k means classification method
Does the Metropolitan Police Service CAD data contain rhythms and cycles that reflect the lives of the people of London and the way it is policed.
What is the best CAD data to use to detect these rhythms and cycles.

From previous multiple bivariant correlations of the temporal nature of the number of different class or types CAD incidents I know that CAD incident class 1 "Violence against person" negatively correlates with the acquisitive crime incidents. I also know that there were subtle differences in the patterns of the total number of these incidents and those that had been graded as "I" for immediate - emergency response. The graph above shows lines for the number of incidents that fall into those four categories.

I carried out a k means classification using those four variables with the 365 days of the year as cases (using raw unstandardised data). I asked for seven groups to see if the k means classification could split the days into the right weekdays.

I find the result shown in the graph above quite exciting. I hope you can see why. The classification recognises a clear difference between Saturdays and Sundays and week days. The week days have been subdivided in two groups. I have been through all the dates which appear to have allocated to the wrong group and there is a very good explanation for each. For instance each Monday that was allocated to the typically Sunday group (group1) were Bank Holiday Mondays. Every other "misallocation" had a weather or holiday related explanation.

I have tried the classification with standardised data and additional and fewer variables but these four variables appear to me to produce the best results.

So I now have even more confidence in the k means classification method, I am convinced I using a good set of data and I am sure that analysing the variations in violent and acquisitive crime incidents in the context of the police response is worthwhile.

.kmz for London and Birmingham crime and disorder

2011-02-10T09:00:00.000Z

I have created .kmz files for the classification shown in the last post. They can be found here

Comparing Crime and Disorder in London and (greater) Birmingham

2011-02-09T17:55:00.000Z

What I am doing here is creating a classification by adding West Midlands police neighbourhoods (mostly Wards) to the MPS wards. The first map is of the West Midlands police area - Birmingham is in the centre, Coventry to the east and Wolverhampton to the west. I am grateful to Andy Brumwell, from West Midlands police for providing me with the shapefile.

Google Map crime and disorder .kmz

2011-02-07T17:06:00.000Z

I have created a .kmz file to be loaded into Google Earth to create the map above. It can be found here. Enjoy.

London Wards December 2010 crime and disorder clustered, classified and mapped

2011-02-07T14:11:00.002Z

Over the weekend I have been trying to improve on my the map in my last post from a London perspective. London was clustered in a group that I classified as having robbery and burglary problems due to where the cluster centres were for that group. In fact London does have robbery problem according to the figures that were used in the K-means clustering method but the burglary figures showed that London had a burglary problem that was about average. It was the small force of Bedfordshire that was clustered with London that does have a big burglary problem and a also a robbery problem that pushed the centre of the cluster in the direction of burglary.

I have used the same methodology as in the last post to create a classification for London shown above but without dividing numbers by police officers and staff numbers. I have excluded three Wards, Heathrow Villages, St James and West End for two reasons.

St James and West End Wards are not shown in the published tables, they are split into eight sub-wards with no information about their locations.
Heathrow Village is Heathrow Airport that clustered uniquely unsurprisingly, St James and West End always cluster together with a far higher crime rate than any other wards. Removing these gives a better scale to the other Wards.

I have been experimenting using the December 2010 crime statistics shown on the official Metropolitan Police website to split up the other crime category into, criminal damage, drugs, theft, etc. The problem with doing this is that it decreases the weighting of the burglary, robbery, etc crimes and ASB in relation to the other crime category. To me the choice of restricted crime categories and ASB reflects the importance of these crimes are to the police and the public. By keeping to those categories and weighting them equally in my clustering I do not have justify anything. Soon as I start making my own selections I have to justify my choices.

Back to the map. In broad terms it shows what I expected. There are a couple of surprises,

the location of the boroughs showing robbery problems, though having checked these do include personal and business robberies; my previous analysis has been exclusively on personal robbery.
the variation in vehicle crime numbers.

It has to be borne in mind that December 2010 was an unusual month with the heavy snowfalls. I need to analyse few more month to see how stable this classification is.

Mapping and Classifying variations Crime and Disorder in forces in England and Wales in December 2010 or variations in recording practices?

2011-02-04T16:51:00.004Z

In my last post I said that there are fundamental problems with the latest UK crime mapping site. I do not like to be negative about things that people have worked very hard to produce so I created the map above to illustrate the problems and benefits of this initiative.

Firstly how did I create the map?

I downloaded the neighbourhood data for all the forces (unfortunately the Sussex spreadsheet was empty) and created a force total of the five categories of crime and one category of anti-social behaviour for December 2010.
I attempted to compensate for the different sizes of the forces by dividing the counts in the six categories by the latest published total police officer and staff numbers for each force. This has its problems (and benefits which I will argue elsewhere) but it is better in my opinion than dividing by resident population.
I then wanted to cluster the data giving equal weight to each category (see my previous posts regarding clustering) so I standardized all the data by transforming them into Z scores. This shows the standard deviation either side of the mean creating positive and negative figures.
I then decided the best probable number of clusters by using the Ward's Hierarchical method in SPSS 17 and choose 6. I then used the K means clustering method with the six clusters.
The K means method allows a better understanding of what factors has influenced the clustering, allowing a classification of the clusters to be reliably undertaken. This classification is shown under the map above with a table showing the scores for the cluster centres.
Now this is where I need help. I cannot find a shapefile for police forces in England and Wales. Edina has a shapefile for police force basic command units for England and a seperate one for Wales. I had to do a bit of editing to the .dbf file to show my classification at force level using ArcGIS.

Some very interesting groupings have resulted and vast differences in the recorded level of crimes and ASB has become apparant.

But what are the data I am classifying and mapping here?

To be honest you I do not know, I could find no documentation to explain. Now I am assuming that the crimes are police recorded crime in Home Office Crime Types but it would not surprise me to find out that there have been problems in ensuring that every force is submitting exactly the same types of crime. The chances of that being uniformly the same is a lot higher than the Anti-Social Behaviour figures, which I am assuming are from the police incident recording database and would not surprise me if they differ significantly in content from force to force.

So that is the number one fundamental problem - what is being counted, how is it counted and is it the same through out the country?

New crime mapping site - impressive but with fundamental problems

2011-02-02T14:44:00.002Z

The new UK Internet Crime Mapping site was launched on Monday. It has received so many hits, reportedly 18 million an hour in the first few hours that I have had to wait until this morning before I managed to connect to the site.

I am torn between being impressed and dismayed, between thinking this is a leap in the right direction to thinking that the people who have designed this have not a clue what they are doing; between applauding the liberating of police data to castigating the fact that data without context and provenance cannot be treated as information.

First the positives.

Innovation and development in public services require monetary investment and risks to be taken. It requires people in power to say `yes' to ideas rather than the safe 'no'. It is the way improvements and advances are made.
Even though there are many problems with this site I think it should be seen leap in the right direction because it signals the acceptance of an underlying principle that the public have a right to know, in detail, what police are doing in their area through medium of police collected data.
Connected with that is treating the public as grown up people who are capable of dealing with the harsh realities of life. This should lead to an understanding what the police know about the policing problems in the area thus enabling a more informed partnership between police and the public to tackle those problems.

The visualisation of data is new, interesting and works smoothly at various levels of resolution. Technically it is interesting; it appears to work on a grid system of squares of I estimate 50 metres by 50 metres. The point shown at the highest resolution is on a road within that grid square closest to the centre (some grid squares seem to have more than one point at the highest resolution, I think this where two distinct post codes or street name can be identified - interesting algorithm). Everything georeferenced to a location within that grid square is shown at the point or points. The visualisation is of circles with the lowest resolution showing a circle with about a 5,000 metre diameter. As you zoom in and out the circles seamlessly change in size as the grid squares are subtracted or added. Different crimes types and antisocial behaviour counts are shown for the month of December 2010. The difficulties of counting across police borders appear to have been solved. There are links to Safer Neighbourhood Teams, with information about crime appeals, crime prevention, police Ward meetings and Neighbourhood Watch Schemes - all impressive.

I will leave the negatives which are fundamental and numerous until next time.

The British Crime Survey intial crime victimisation questions

2011-01-24T00:54:00.001Z

I am going to try to explain something quite complex in this post but it is important to my research. It relates to the issue I discussed in my last post; the difference between the public's labelling of crime based the everyday use and common knowledge of the English language and the legal definition of criminal offences and how they are counted and categorised by the Home Office.

I have decided the best way to explain this is by reference to the methodology used in the British Crime Survey (BCS). The BCS is a survey of the general adult population of England and Wales that selects a sample of about 44,000 people a year to represent the total population about 44 million. The survey includes asking questions about whether the selected representative of the selected household has been a victim of crimes. Now because the interviewer can not assume that the person being interviewed has specialist knowledge about criminal offences within the England and Wales legal system there is a standard questionnaire that asks questions in everyday English.

A little bit of an aside the BCS in a masterpiece of covering just about every eventuality as one would expect from a piece of work that has been going almost 30 years and undergone numerous rigorous reviews of its methodology.

One of the first issues it covers is whether the person has moved in the relevant time, that is the last 12 months. This affects household victimisation, thus the slightly odd set of questions that are asked early on about crime types. These are listed in the table shown below.

The BCS dataset lists the numbers as options for variable `Crimtype' with the BCS data dictionary giving the names shown in the second column. I have examined the 25 sets of questions that produce answers regarding whether the household or individual has been subject to various events and if so how often. I have then used my specialist crime knowledge to explain the options. The way the questions are phrased allows burglaries to be identified. It is not possible though to differentiate between a theft person and a robbery as violence questions are at this stage dealt with separately to theft offences.

The table above should be read from left to right. The gaps signify where there is no comparable data. Even where there is comparable data there are caveats which I will cover in subsequent posts. What I want to point out is that the initial question (which seems to be to identify potential victims) with a little bit of recoding produces good results, ie consistent with published results. I do concede however that the data set I am dealing with only includes those cases that eventually were deemed to be offences.

Crime, soil, perception, labelling and mapping

2011-01-21T16:54:00.003Z

Link

I have been a little bit silent on my research for a few weeks for various reasons but I think it is the right time to try to explain what I am up to.................

There is a phrase up North that is "Call a spade a spade" or another one "Its what it says on the tin". The meaning of course is that the label given to something is 100% accurate and its all you need to know. The point I wish to make is that this is not the case with police recorded crime, which in the UK is the source data for crime maps. As I have discussed previously, for instance, the label `violent crime' is totally misleading. This I argue is at the heart of the public's mistrust in crime statistics. The requirements of police recorded crime to meet legal definitions, Home Office counting rules and groupings into Home Office types result in the label on the outside of the tin not representing what the public assumes to be on the inside of tin. It is not surprising therefore that the public's perception of various types of crime is different to what the police recorded crime statistics show. Then the there is the additional problem of unreported crime.

The question I have set myself is;

Is it possible to produce valid crime data that more closely matches people's perception of crime for mapping purposes?

This is a sentence I have used when trying to explain the the importance of this ambition;

We agree with others (see Jackson and Bradford 2010) that trust is aligned with effective community engagement based on a shared understanding of problems to be addressed, which in turn leads to increased confidence in the police and the criminal justice system.

The problem of producing maps with data that the public do not trust is that it is likely to result in the public thinking that the police do not understand the crime problem. If the public do not think the police understand the crime problem it is logical for them to think that they are not tackling it properly, this inevitably leads to the public losing confidence in the police.

So that is the theory, what data is out there for me to use?

Let me give you an analogy off the top of my head.

When I travel round the countryside I notice ploughed fields, in different areas the soil is different colours varying from a rich dark chocolate through reds, yellows, greys, creams to almost white. That is my personal unsophisticated almost subconscious classification of soils, I am not a farmer, I am a city person, the countryside to me is about aesthetics and ambiance. Now to a farmer soil classifications are to do with what it consists of, its water retention and what he can and cannot grow on it; the soils are graded accordingly. I go into a map shop and decide that I want a map of different types of soil colours in the UK. The shop keeper tells me that there is not such a map but he can sell me one of grades of soils which is just as good because everyone knows the better the soil the darker in colour. I then test this assumption out only to find that it is not always the case. I want a map of what I can see not what requires specialist knowledge to understand. No doubt when grading the soil one of the early scientific observations would have been the colour, so it would be quite easy to produce a map of UK soil colours if the original soil grading data could be accessed.

So the data I am going to use is not the data for specialist that has been accurately sorted and classified - police recorded crime but the aspect of the data that the public observe and label in everyday language. I am going to the early stages of the data collection process.

I will make this clearer in my next post.

Reference

Jackson, J. and Bradford, B. (2010). 'What is Trust and Confidence in the Police?', Policing: A Journal of Policy and Practice, 4,3, 241-248

Geography, Theresa May and Confidence

2011-01-20T11:36:00.001Z

I have suddenly got more confidence in our Home Secretary Theresa May because I have discovered that she has a BA in Geography! A few other interesting facts that make me feel that I know her better if wikipedia is to be believed;

her father was a Church of England Minister presumably of a high church persuasion given her first name and the fact that her early education was at a Convent School,
her maiden name was Brasier, no doubt not pronounced as it reads, but not the easiest of names to live with,
her constituency is Maidenhead which means that she is likely to be close enough to the Cameron's Oxfordshire cabal (who apparantly run the country) to be a part of the incrowd at the weekends.

The latest recorded crime statistcs for England and Wales have been released accompanied by an announcement from Theresa May. The announcement is actually more interesting than the statistics. I can do no better than quoting from the BBC website

Home Secretary Theresa May said that officials were now launching a review of how crime figures are collected and published in an attempt to improve public confidence.

The review would look at ways of cutting costs while making the figures more coherent.

"Any reductions in crime are welcome, however levels are still too high and we know these statistics only offer a partial picture about the level of crime," Mrs May said.

"More needs to be done to bring crime down and we need to take bold action to restore public trust in crime statistics.

"That is why I have asked the National Statistician to lead a review and why we are moving the publication of crime statistics out of the Home Office to an independent body.

"Improving public trust and confidence in crime statistics is crucial if we bare to improve transparency and empower local communities to hold authorities to account."

The review should, and I think will, include crime maps for the public. I think I should, and I think I will try to get involved...............

The Joy of Statistics

2011-01-14T00:01:00.001Z

I will be brief with this post because the link I think is only available in the UK.

I watched this program about statistics this evening on BBC 4 and even though I did not agree about everything it assumed about crime data I found it a very interesting and thought provoking. You can watch on on BBC iplayer following this link

What is Crime? - notifiable offences?

2011-01-09T23:28:00.001Z

I want to write about police recorded crime and notifiable offences. I wanted to give you a bit history to make it all a bit more interesting, otherwise it would have just been "police recorded crime is confined to notifiable offences. They are called notifiable offences because the Home Office requires all Home Office police forces to notify it of the numbers of these crimes that occur each year in their territorial jurisdiction based on Home Office counting rules". The current ones can be found here, which also includes a link to a list of notifiable offences.

You would think the reason why no one has apparently written a paper outlining how the system came about is because it is deadly boring and not worth the effort. I actually think it is a quite an interesting story if I have got the right end of the stick - my sole source is Hansard, the official record of parliamentary proceedings and therefore quite a good source; it is now helpfully on line from 1803 onward.

The first mention of crime in online Hansard is in 1824 in a debate about magistrates and the increased level of committals, convictions and executions. This was of course before the first police force was introduced - the Metropolitan Police in London in 1829. It is apparent from the debate that the moral health of the country was measured by these Crown Court statistics which had been collected for at least 15 years before the debate. In fact in 1828 it was these same source of statistics that was used by the Home Secretary Robert Peel to show that committals had increase by almost a third in the previous seven years to justify the introduction of the Metropolitan Police Service.

Possibly the most scathing attack on government statistics ever was made by Lord Brougham on 3rd of March 1856 (in the House of Lords) when he showed that the Judicial Statistics that had provided evidence for Government policies for at least the previous 40 years were worse than useless. Even more embarassing was the fact that the equivalent French statistics based on Guerry's work were almost perfect in comparison, at least according to Lord Brougham. The speech included the following;

Hitherto I have spoken of offences and offenders as recorded in the proceedings of courts, whether of trial or of police; but it is of the greatest importance to ascertain as far as possible the number of crimes which never reach any court: nothing can be more essential to the formation of an estimate touching the state of crimes in the community, and the action of the law in detecting and in preventing them.

His long speech came to an end with a very long list of judicial statistics that should be collected in the future including crimes that do not reach the courts - police recorded crime. In reply the Lord Chancellor said that Lord Brougham suggestions would be looked at seriously.

Exactly a week later on 10th March 1856 the Police (Counties and Boroughs) Bill was debated in the House of Commons. This became the Counties and Boroughs Police Act 1856 that made it compulsory for any County or Borough that did not already have a police force to introduce one. It also created what is now Her Majesties' Inspectorate of Constabularies to report regularly to the Home Secretary regarding the efficiency of each police force (see here). It is known that the first police recorded crime statistics were collected for the year 1856 so there seems to have been some co-ordination behind the scenes for the Home Office to draw up a list of notifiable offences.

What is crime? - recordable offences?

2011-01-03T22:50:00.003Z

I know I am feeling better, because I have an urge to write something that probably is no interest to anyone but myself. I went on a short trip to Thailand and Vietnam just before Christmas and managed to get home just before the snowfall landed on Heathrow. Unfortunately I managed to eat something that upset my stomach quite dramatically and after that I picked up a virus from IBM. Not a computer virus - a man flu virus via my son in law who works for IBM. That slowed me down a bit but unlike the previous bug it did not stop me eating so I did enjoy Christmas, thanks for asking, but I have not felt up to doing the work I was planning. Thus the absence of posts.

Now what has got my mental juices flowing again? Its the problem of defining "crime".

The first thing to say that I am approaching this from my knowledge of the English and Wales legal system. There are many other legal systems out there...................

In our legal system there is a fundamental principle that is even more basic than "innocent till proven guilty" which is often misquoted and misunderstood -you are guilty the second you commit an offence but for the purposes of the criminal justice system you are regarded as innocent until due legal process proves you guilty beyond reasonable doubt. The principle I am referring to is that a crime has to be defined (usually) by legislation. The criminal law assumes everything is legal that is not specifically enshrine in Common Law, defined as a criminal offence by Act of Parliament, or arguably a body permitted by Act of Parliament to create offences - thus local bylaws. Criminal Law has its own courts, rules and sanctions.

Travelling in parallel with Criminal Law but very much on its own tracks is Civil Law. Civil Law has all sorts of weird and wonderful things but it does not have offences and crimes. It can be said that criminal law deals with absolutes that if put into questions should have a yes or no answers. Civil Law on the other hand deals with gradations whose answers are more fuzzy - maybes and depends are better answers. And of course the burden of proof in civil cases is on the balance of probabilities that reflects these differences.

But I digress. What I am trying to show is that with a legal system such as the English and Wales one the broadest legal definition of crime is an offence under criminal law. This of course includes parking offences, speeding fines, littering, drunkeness in a public place and other offences that we generally regard as being committed by normal members of the public, not criminals.

To have a criminal record you have to have been convicted of a recordable offence. Once you have been convicted of a recordable offence you are allocated a unique number (and letter) which in the days when the record was a paper file in the Criminal Records Office at new Scotland Yard was known as a CRO number. That's why when you get charged with a recordable offence you have your fingerprints and photograph taken. Now in a more modern world the record is no longer paper it is computerised on the Police National Computer (PNC) and you are allocated with a PNCID instead of a CRO. There are of course still people with both. And nowadays DNA is also taken as an identification. Details of what are recordable offences are here. Recordable offences are basically offences that have a term of imprisonment as a sanction plus a number of specified offences.

Now this is the confusing bit; the official crime statistics - police recorded crime - is a system that is quite separate from the recordable offences discussed above and it is based on notifiable offences. But that is for the next post.

Shedding more light on the Dark Figure

2010-12-10T18:16:00.000Z

This is yet another Venn diagram. I have been a little addicted recently. This is taken from my 15 minute presentation entitled "Shedding more Light on the Dark Figure" which I gave on 7th December 2010 to the Crime Survey Uuser Group meeting. They have published it on the web along with the other presentations. They can be found here. Mine is the 2pm time slot.

Perceptions of Crime

2010-12-04T21:47:00.000Z

I have been studying my Venn Diagram in the post one before the last decided that I can improve on it. This is not perfect but it is better. I actually like the diagram without the labels. Perhaps I should enter it for the Turner Prize entitled "Perceptions of Crime". OK.................. here it is with labels.

Linking Crime, Incident, BCS and Police Recorded Crime data sets

2010-12-03T17:38:00.001Z

Posts to do with the the Metropolitan Police Service CAD and CRIS databases have been quite popular so I thought I would share a flow diagram that I have just created of how the data arrive in them. The other two data sets I am interested in - the British Crime Survey and Police Recorded Crime are also shown.

Overlapping Crime Domains

2010-12-03T13:12:00.000Z

This is my latest Venn diagram trying to explain Crime: how it is measured/recorded and how it is perceived. I think it is reasonably self-explanatory. The interesting bits are the overlaps of the various domains. I am approaching this diagram from three different angles;

Shedding more light on the "dark figure" a talk I am giving to the Crime Survey Users Group on Tuesday 7th December.
Explaining the utility of police incident data in augmenting the other crime measures (BCS and police recorded crime) particularly regarding violent crime for a paper that I am writing, and
My ongoing interest in describing the fear of crime.

The early history of crime maps with modern visualisation

2010-12-02T23:05:00.000Z

I would like to draw your attention to a high quality paper (I have just discovered) that should be of interest to everyone interested in crime mapping. Its got it all, it has the history, the maps, the visualisation and a statistician who can write good English without feeling the necessity to resort to Ancient Greek lettering to make his point except where it is truly necessary.

This is an example of one of many maps within the paper. You will have to read it to find out what it is all about. The free .pdf can found here.

Temperature and violence

2010-11-25T10:51:00.002Z

This blog entry is dedicated to my colleagues in CASA, especially Andy Hudson-Smith, Ollie O'Brien and Fabian Nuehaus.

Yesterday I collected data from CASA's weather station for the 2009 calendar year. I have collated it for analysis and these are my first results.

The reason why I have written a blog entry for three weeks is that I have been writing a paper entitled `A Geographical and Temporal Analysis of Violent Crime in London in 2009' which hope to submit to a journal for publication. This has switched me on to the characteristics of violent crime that are different to other crimes. One of things I noticed was the affect of events such as New Year's Day had on violent incidents but also the weather.

I thought I would look into the weather aspect a bit further and I am able to thanks Andy and Ollie. The graph above does not look to promising. It shows the maximum temperature for each day in 2009 on the y-axis and the number of incidents that were allocated a code 1 class 'violence against person` and graded as an 'I` or emergency call on the x-axis. The result shows a very weak positive correlation. Nothing to get too excited about there then.

This where I thank Fabian for helping me to regard everything in police data as monitoring a machine (or a body) that has rhythms and cycles. Police incident data has rhythms and cycles that you can set your watch and calendar by.

You can see that number of calls is affected by the day of the week. Working on the hypothesis that the rhythms and cycles of London that are reflection of people's work/leisure routines have a greater influence than weather I negated much of this variable by just using Saturday data. I tried various combinations of weather data including average daily temperature and various different CAD codes in combinations or divisions. The best correlation I found was with CAD code 1 emergency calls and maximum temperature.

An R squared of 0.4417 means that temperature is not the only influence on violent incidents, holidays, celebrations, special events all play their part. These in some cases may be a contributory mechanism by which temperature is an influence, (eg hot May Bank Holiday weekends) and in others may be a negative influence (eg colder New Year's day( not on a Saturday in 2009) or Halloween). In any case the R squared value represents a correlation of 0.6646 which is generally regarded as significant in social sciences.

First Global Crime Map?

2010-11-05T16:57:00.002Z

To go to the interactive map click here

I stumbled across this website. I like it for all sorts of reasons. The main one is because I think, in its way, this may be the first interactive global crime map. OK prove me wrong.

Venn Diagram with the example of Burglary in Dwelling

2010-11-01T23:10:00.002Z

Venn Diagram with the example of Burglary Dwelling in London in 2009

In the last posting I showed a Venn diagram of the relationship betwee CAD and CRIS. Today I have amended it slightly and populated it with numbers relating to Burglaries in Dwellings. You will notice that from the figures almost all the burglaries are repoted via a CAD incident. This is what I would expect as burglaries in people's home are a priority.

Drugs - Harm or Danger, or both?

2010-11-01T09:25:00.001Z

I would just like to briefly comment on the current headline story "Alcohol 'more harmful than heroin' says Prof David Nutt" This appears to be an update on a previous Lancet Article in 2007 "Alcohol 'nearly as harmful as heroin'"

I know Prof. Nutt due to working on a Foresight Project Brain Science, Addiction and Drugs back in 2005 and the co-author Dr King who advised on a some drug trafficking research I was part of.

The aspect I would like comment on is the assessment of harms that is the basis of the research. From my understanding the same methodology could be used to come up with the headline "Sugar more harmful than Guns" or "Potatoes more harmful than Arsenic" or "Toasters more harmful than Nuclear Submarines".

In my opinion an assessment of harm should not be isolated from an assessment of risk. Prior to the Misuse of Drugs Act 1971 there were various Dangerous Drugs Acts.

We should be assessing what is dangerous as well as what is harmful.

The Crime Record Information System CRIS discussed

2010-10-31T23:31:00.001Z

Today I am discussing the Metropolitan Police Service Crime System, CRIS. I have just done a number of Google searches allow me to refer you to MPS or other sites that give background information about CRIS but there are none. So I will have to try my best to supply the relevant information.

My major first hand experience of CRIS was running a Crime Desk as Sergeant at Chiswick Police Station and then Hounslow Police Station when the two Divisions amalgamated as part of Borough Policing at the beginning of 1998. I went from a rather poor user of CRIS before the job to an expert in under 6 months due to working on it 7-10 hours a day. CRIS has changed a bit over the years but not significantly from what I am able to tell from my searches of the system.

Unlike CAD, which is a system that is simple, elegant and clever due to it being designed from the start to maximise the strengths of computerisation, CRIS is none of those things for one basic reason, it totally copied the paper based crime reporting system that proceeded it. Let me briefly take you through the history.

When I joined the police and for over ten years the way I would record and investigate a crime was by completing a paper crime sheet with a pen. These crime sheets had been developed over the previous 150 years of the Metropolitan Police history and were very good in ensuring that there were places to complete all the necessary details of the crime, show the investigation, suspects, people arrested, supervision, Home Office classification etc, etc. These were in a series of boxes and new sheets could be added as necessary. Different crime books were maintained. Major Crime, Beat Crime, Vehicle Crime, Burglary, Drugs and sometimes Robbery.

This is another very important point. The purpose of the paper record was to record crime and its investigation; and be counted for official crime statistic purposes.

All that happened with computerisation was that the paper system with it series of boxes were turned into a computerised system with the same boxes which every police officer had to update like they were completing a written record. No thought was given to how computerisation could actually make policing more efficient and effective. An opportunity was lost to free up police officers time, for instance, by having specialist civilian operators taking reports from police officers at the scene of the crime . No thought was given to making the investigation details not free text but a series of searchable structured fields, or how the information could be used for intelligence and mapping purposes. The Metropolitan Police is still suffering from these short sighted decisions.

CRIS then is the definitive record of crime recorded by the police. Some of those crimes are recorded as a result of police incidents recorded on CAD. Others are reported over the telephone, via the Internet, at the front counter of police stations or discovered by police, for instance after arresting someone, investigating another crime, etc.

Now this is where things get slightly complicated. There is a difference between that which is recorded on the CRIS - given a CRIS number - and that which is counted towards the official crime statistics. There are three main reasons for this;

The CRIS record is initually of an allegation, after investigation and supervision the correct classification is given to the record. This is where good "housekeeping" can "legitimately" reduce the number classified as burglary, robbery etc. - the priority crimes.
In the MPS all through my crime recording and investigation days were very straight forward with their crime recording - all allegations went in one system and were classified as crimes unless there was good evidence to show it was not a crime. I believe it was the practice in certain forces to keep the crime rate down and clear-up rate up for a dual system to operate where only crimes that were supported by good evidence went into the official crime book, the others were recorded separately and were only transferred across if more evidence was forthcoming. It seems that the spirit of this has been adopted by the MPS by creating something called the crime incident. This is to ensure that nothing that turns out no to be crime can possibly be counted in the official crime statistics. For instance, where the occurrence of a crime depends on the outcome of forensic tests; a CRIS record is created not as a crime but as crime incident so that a 7 day (it may have changed) classification rule does not apply.
CRIS has filled the gap in the recording of non-crime incidents, most notably Domestic Incidents, known as non-crime domestic incidents. This means that nearly all CAD domestic incidents have a CRIS number; this only sometimes means a crime has been recorded.

Simple Venn Diagram Showing the relationship between CRIS and CAD

Police Incident Data Set Discussed

2010-10-29T19:24:00.005+01:00

Police Crime Incidents involving a firearm in London in 2009 at Borough Scale

Legend - Number of Incidents

Police Crime Incidents involving a firearm in London in 2009 at Ward Scale

Legend - Number of Incidents

In this post I am going to start to tackle a difficult question but one which is very important to crime mapping and my research. You may find some aspects of the answer surprising, I do.

The question is "How do my two data sets, the police incident data set - Computer Aided Dispatch CAD and my crime data set - Crime Recording Information System CRIS relate to each other?"

This is a very important question as internationally the United States "crime" mapping systems for the public appear to be almost exclusively based on their incident or call for service databases whereas in the UK the crime database is exclusively used.

The first thing to say is that the two systems I am taking my data from - CRIS and CAD were introduced into the Metropolitan Police long before computerised crime mapping became common place. The two systems where introduced for different purposes. So a bit of history.

When I joined the Metropolitan Police in the late 1970s computers where just being introduced but the police incident recording system was still 90% non-computerised. There was a dual system of central and local. Emergency calls (999) were routed to Information Room at New Scotland Yard who would assign local units with mainset radios to calls. The local police station communications room would be notified through a Message Switching System (MSS) which printed out the call onto a printer. This call would be placed on the A5 pad that had been started at midnight that day and given the next number in order. When the call was dealt with the result would been written on the sheet. The MSS system would also be used to send and receive messages from other police stations to be dealt with - bail enquiries, Please Allow An Officer To Call On (PAAOTCO), court warnings etc. In addition the local police station would have the non-emergency telephone calls routed to it (which were quite often emergencies). These would be manually written onto a message pad with the nature of the call the location, details of the caller, time and unit(s) assigned, and when the call was dealt with write the result. This was often in local code abbreviations the actual meaning of which usually had a correct cover story but often a (politically) incorrect everyday usage. The Comms officer would use the local police station radio system to communicate with police units and be responsible for monitoring incoming calls. Back then as now there was a problem with capacity so the telephone operators would divert everything that was not needed to be dealt with by the uniform police response team on duty to other police offices.

This is the first very important point. The police incident system does not and never has recorded all calls from the public.

In the early 1980s the CAD system was introduced, first in New Scotland Yard Information Room and gradually over the next few years to the Police Stations control rooms around London to replace the paper system. Numbers replaced the words relating to the nature of the incident and the result. A graded response system was also introduced. This is how I knew the system when I last used it operationally in late 2004.

So for the next bit of history, bringing it up to the present day, I rely on second hand information from what I have learnt through conversation, MPS documentation and these two website sources. The first is from the MPS website here and the Wikipedia site here, which is accurate (in the main) but needs updating as the information is now at least two years old.

It appears that capacity has again been the problem which has meant that CAD system being supplemented by the Call Handling System and the introduction of three Information Rooms around the MPS to replace the Police Station Control Rooms and the New Scotland Yard Information Room.

Call numbers are confusing. The MPS claims there are about 6,000 to 10,000 emergency calls a day and about 15,000 non emergency calls, a total of 21,000 to 25,000. The confusing bit is that within the emergency calls there are non-emergency incidents and visa versa. The result is that in 2009 there were about average of about 10,000 CAD incidents a day with a low of about 7,000 and a high of about 15,000. Of these about a third were dealt with as true emergencies - attend as quickly as possible, a third not as emergencies at all - attend when police can or sort it out by other means, and a third as semi-emergencies, attend within the hour, if possible. This means that about another 10,000 calls are dealt with by the Call Handling System (CHS).

So in my analysis I have to be aware that if I confine myself to CAD data, which I have, and not delve into the into the CHS data, which I have not, then I am dealing with a subset of the total demand on police. I am reasonably confident, and I am working on this basis, that I am analysing a data set that has been selected by rules that are consistently applied throughout London over time. This I am assuming has produced a complete data set of incidents it is necessary to be referred to officers on duty to deal with. The filtering by call grading is a secondary part of this process.

In the next post I will discuss CRIS and explain the maps at the beginning of this post. I think I has written enough for now.

The deficiencies of Home Office Crime Type 1 exposed again

2010-10-28T23:29:00.000+01:00

Home Office Crime Type 1 -Violence Against the Person in London in 2009 shown at ward level

Legend - number of crimes

In this recent post I discussed Home Office Crime Type 1 -violence against the person and the fact that only about 40% of crimes involving firearms in London in 2009 were included in that category. The above maps shown the geographic distribution of such crimes at ward level. I obtained the data from the Metropolitan Police website here

The correlation between locations at ward level of my data set of crimes involving firearms and HO type 1 is 0.53. This is a low figure given that the average member of the public would expect that in most cases that crime involving firearms should be included in Home Office Crime Type 1.

"Do the recorded victims and offenders of police recorded crimes involving firearms live in the same places?"

2010-10-28T10:50:00.000+01:00

The question I am addressing in this post is "Do the recorded victims and offenders of police recorded crimes involving firearms live in the same places?" Then I go on tho answer similar questions relating to deprivation and crime venue.

First a few details about my data set. I have been using the police recorded crime data set in my recent posts about firearm related crime. "police recorded" means that it is only a subset of the total occurrence of crimes involving firearms. I can make no guess, comment or observation on whether this subset is typical or representative of the total. Thus "police recorded crime" part of the question. Police are good at recording victims and their addresses. There are only 8% of victims without a grid reference address. The details of a suspect or a person who has been arrested for an offence, an accused is far less complete due to police only detecting a percentage of crimes. Thus the question having "recorded" twice in it. I am using offender as shorthand for suspect or accused.

Next thing is; I am not looking at individual crimes, I am looking at all the crimes together. So if offenders and victims do live near each other it does not mean that they have been involved in the same crime, though it is possible. What I have done is correlate the location counts of victim and offender addresses at the different scales I have been mapping. Borough, Ward and Grid Square. The results are as follows;

Borough - 0.95 correlation meaning that the boroughs with high number of victims have high number of offenders - an almost perfect correlation
Ward - 0.68 correlation meaning that wards with high number of victims have high number of offenders - a significant correlation
Grid - 0.30 correlation meaning that grid squares with high number of victims have high number of offenders - but this is not a strong correlation meaning there are many exceptions to this trend.

What can be inferred from this? It can said to a high degree of confidence that victims and offenders live in the same general areas as it each but not necessarily in exactly the same places. In London millionaires can live close to those claiming benefits so there is diversity in housing even at a grid square scale.

The table above gives the correlation results when victim and offenders address locations are compared with the Index of Multiple Deprivations (IMD) scores. I have discussed the IMD in detail on this blog so I will not explain further here. Just to give a word of caution, the spatial scales I am using are not those that the IMD was calculated at so there is a degree of inaccuracy that could be present at Ward and Borough level, though not much as they are adding LSOA scores together and taking the mean; the grid square IMD score is very much an approximation as I have discussed previously. This probably explained the Grid Square correlations show a higher correlation for victims where the other two show a higher correlation for offenders

Now this table showing correlations between the victim's and offender's addresses (in bulk) and the venue of the crimes (in bulk). It show that on a local level - grid square and ward the location of crimes are nearer victim's homes but at a slightly more general level - borough the trend is reverse. All these correlation are high though.

Locations of Firearm Crime Perpetrators' homes

2010-10-27T15:53:00.002+01:00

Recorded Crime in London in 2009 involving firearms showing recorded accused/suspect address at borough scale

Legend - Number of Suspect/accused addresses

Recorded Crime in London in 2009 involving firearms showing recorded accused/suspect address at ward scale

Legend - Number of Suspect/accused addresses

Recorded Crime in London in 2009 involving firearms showing recorded accused/suspect address at grid square scale

Legend - Number of Suspect/accused addresses

Recorded Crime in London in 2009 involving firearms showing recorded accused/suspect address at zoomed grid square scale

Firearm Crime London - more witness and victim maps

2010-10-26T09:10:00.001+01:00

Recorded Crime in London in 2009 involving firearms showing recorded victim and witness address at borough scale

Legend - number of witnesses and victim addresses

Recorded Crime in London in 2009 involving firearms showing recorded victim and witness address at ward scale

Legend - number of witnesses and victim addresses

Following on from the last posts where I mentioned the influence of firearms crime goes beyond the recorded victims to at least the witnesses and probably further. See my discussions about fear of crime and signal crimes/events. The maps above showing both witness and victim addresses in London give slightly different maps.

Firearm Victims in London Maps

2010-10-25T21:56:00.002+01:00

Recorded Crime in London in 2009 involving firearms showing recorded victim address at borough scale

Legend - number of person victims

Recorded Crime in London in 2009 involving firearms showing recorded victim address at ward scale

Legend - number of person victims

Recorded Crime in London in 2009 involving firearms showing recorded victim address at grid square scale

Legend - number of person victims

I think these are self-explanatory. Just two things to say. Some crimes have businesses as victims, these are not shown, just the people victims. In my experience if the victim has beeen acting as an employee the work address is often given otherwise the home address is recorded, sometimes both. The second piont is I have only mapped victims of London crimes who live in London.

Firearms,victims, witnesses and splitting hairs

2010-10-25T15:07:00.001+01:00

For a crime to have occurred there has have been an offender or offenders. Offenders are usually human beings though a corporate bodies can be offenders in rare cases. Because people occupy space and time (and so must corporate bodies because of laws regarding jurisdiction) it can be said that a crime always has a location and a time, that is, where the perpetrators are when the crime are committed. As I have discussed in a recent post there can be more than one location and time if an offence is a continuing one or where there is a victim or victims who are at different locations or where the crime, though "victimless" is one which the perpetrator can affect remotely.

Now why am I splitting hairs like this? Well the crime commission process is extremely important in the analysis of the occurrence of crime. The more precisely the factors that are involved in the occurrence of crime are understood and defined the more precise the resulting analysis will be.

For a crime to occur therefore there has to be three factors offender(s), location(s) and time(s). This is a slightly different angle on the Problem Analysis Triangle from Problem Oriented Policing (POP) shown below

This has been taken from the Centre for Problem Oriented Policing website that can be found here

Above is my graffiti emendations. Firstly offenders commit offences or crimes not problems, secondly as I have shown throughout this blog time is extremely important and is always present (forgive the pun) with place. Now the third factor which is not always present (eg driving offences) are; a victim (eg theft, assault, etc), a target (eg goods to steal or damage) or a commodity (eg illegal drugs, firearms etc).

So yesterday I showed location. Today in the graph above I show the gender and age characteristics of the victims and witnesses of crimes involving firearms. This brings me on to another concept. Who is affected by crime? The victim obviously, but probably those that witnessed it as well if it was a violent or frightening event as many crimes involving firearms are. That is the reason why I have shown both graphs. The reason for the jagged peaks is because these are often estimated ages and these at often estimated at 5 year periods.

Zooming in on Firearm crime locations in London

2010-10-24T15:30:00.000+01:00

Recorded Crime in London in 2009 involving firearms showing recorded Ward location

Legend - Number of crimes

Following on from the previous post I am showing the same data as the second at council ward and grid level. What I forgot to mention is my classification of quantities uses equal intervals with 0 excluded for all the maps; I have also tried to use the same clours so that they can be compared.

Recorded Crime in London in 2009 involving firearms showing recorded grid square location

Legend - Number of crimes

Recorded Crime in London in 2009 involving firearms showing recorded grid square location - Zoom detail

The interesting aspect of these maps to me is the borough of Haringey (top centre of zoom grid map). The borough map published yesterday showed that Haringey is not in the top four boroughs ranked by number of crimes but it has a ward that is ranked number 2 and a grid squre that is ranked number 2 which is not located within the second ranked ward!

Firearm Crime, the problem of counting and location discussed

2010-10-23T14:26:00.003+01:00

Recorded crimes involving firearms in London in 2009 showing Investigating Borough

Legend - Number of Crimes

Following on from the previous post I am answering the question "Where do these crimes occur?" I am in danger of immediately confusing you when answering this question. There is an assumption with crime mapping that a crime happens in a specific place at a specific time. This of course is not always the case. The commission of a crime can happen over a long period of time in many different places. Good examples are possession of illegal drugs or driving a stolen car. The location and time of a crime are usually recorded where they are discovered. What about internet related crimes, where are they committed, in cyberspace or the location of the victim, if there is one, or the location of the perpertrator, if that can be discovered? So one way of answering the question is which police "own" the crime and are responsible for investigating it. This is usually to do with physical location of the crime but when this is not easy to establish or where there are many locations then the Home Ofiice rules apply - which defaults back to where it has been discovered or first reported to police if no other rule applies.

So the first map above answers the question by saying which borough police are responsible for investigating and recording the crime. This has been worked out based on a data set of 5587 crime incidents. You notice that in the previous post the total of offence is 5648 - a difference of 61. This is to confuse you further and to further illustrate that numbers are rarely simple with crime counting, this is because on rare occassions the same incident is recorded as more than one offence. This is unusual at the initial reporting stage (which my data set is obtained from) but happens quite often when a crime has been solved and further offences can be added, thus helping the clear-up figures, but that is a different story.

Recorded crimes involving firearms in London in 2009 showing recorded location

Legend - number of crimes

Whichever way the crimes are counted London is fortunate in having a low level of crime which involve firearms. The most dangerous boroughs appear to be Lambeth, Southwark, Lewisham and Hackney.

Firearms and violence

2010-10-22T18:59:00.002+01:00

Yesterday the Home Office produced another publication updating the Official Crime Statistics. These are based on the British Crime Survey and Police Recorded Crime. The publication can be found here. As usual this is a high quality document. I thought I would provide some alternative figures regarding violent crime because, as I have discussed before, a lot of violence does not fall within Home Office Crime Type 1 - violence against the person.

The table above shows all crime recorded by the Metropolitan Police Service, (which polices London) in 2009 that involved a firearm in the offence. On intial classification there were 5648 such offences. The tables show exactly what the offence is and which Home Office Crime Type they are categorised in. I think most people in the UK would think that offences involving firearms are the most violent of violent crimes. OK there may be a few technical offences in this lot that do not actually involve violence but the vast majority are really nasty crimes. Only about 40% are included in Home Office Type 1.

A good use of Twitter

2010-10-18T22:12:00.001+01:00

Police have a difficult job there is no doubt about that. One of the difficulties they have is showing what an effective job they are doing day to day. I am afraid that by trying to show, often falsely, that crime and disorder is low they are doing themselves no favours because it undersells their essential value to vast areas of the UK. The fact is confidence in police is tied up with the public's demand on them. High demand means the public think police are there to help them and that the police have the ability to meet those high expectations. As a police officer I have dealt with all sorts of incidents that can be argued are not core policing duties but I was brought in the British Policing tradition that the police are essentially there to serve - thus the word Service in Police Service. Even the application of force, arrest etc. is done with motive of service to society as a whole.

This is where the crime maps that the British Police produce have got it wrong because they do not show the police activity that goes on. That is why I think Chief Constable Peter Fahy of Greater Manchester Police has got it completely right by using Twitter to publicize the volume and variety of incidents in a 24 hour period. The results can be found here and here.

It appears Greater Manchester get about 3000 incident calls a day. As a retired Metropolitan Police Officer from London I think it is my duty to point out this is about a quarter of the calls the MPS deal with in a day.

Populating the grid squares

2010-10-15T17:30:00.001+01:00

I am reasonably happy with the results shown above. This is populating my grid squares with human residential population data obtained from the Office of National Statistics 2001 census. I have used a similar technique to that which I discussed with IMD scores in my last post with a few variations. Firstly I have used population data from the smallest census unit, the Output Area(OA). There are 24140 OAs in London which is a remarkably similar number to the 26116 grid squares, but OA are based on a residential poulation of about 300 people and therefore are of vastly varying sizes. I have experimented using average numbers of people but this gives a highly inaccurate result because there is a bias to OAs with large spatial areas as the grid squares falling within them all having the value of the OA which instead of being counted once will be counted many times. So instead I have gone for average density (remembering that to go via the average number of people and average area rather than averaging the individual densities) to counter-act the area bias. When added all togeather there is a resulting error of about 10% too high a population and about 30% too high an area resulting an average density in London of about 37 people per hectare rather than about 47 which the actual data supply a resultant error rate of about 20%.

This is acceptable to me because (a) the 2001 data is dated (b) the relationship between residential population and crime and disorder I have already argued on this blog is not straight forward.

The naive smoothing is there (this is what causes the error) but to me it is intuitively more of an advantage than a disadvantage.

What is possible from the Census Data is use different age groups in the population. These are what the following maps represent.

0 to 15 years old inclusive

----------------------------------------------------------------------------------

16-29 years old inclusive

------------------------------------------------------------

30-59 years old inclusive

---------------------------------------------------------

60 years old and older

Mapping scale and IMD again with a difference

2010-10-15T15:08:00.000+01:00

This is a map of London displaying the the Indices of Multiple Deprivation 2007 scores (higher the score -purple - the more deprived). These are aggregated at Lower Super Output Area scale. I have used a geometric interval classification in ArcMap as it seemed to provide a better split of colours on this and other maps in this post. The explanation for geometric classification, such as it is, is here. This recommends displaying a histogram of the distribution so here it is.

The point I with to make is that the map that is created shows LSOAs next each that are either end of the deprivation scale. This can be seen in the map below;

Now I need to transfer the IMD scores from LSOA to my grid squares. There are 4760 LSOAs in London (5 more in the City of London) and there are 26116 grid squares so many grid squares fall completely within a LSOA: these in my calculations are allocated the IMD score of the LSOA it falls within. Others fall across two all more LSOAs: these in my calculations are allocated the average of the LSOA scores the grid square intersects. The maps below show these intersection counts, the maximum is 9.

The result of this is what my colleague Dan called naive smoothing; meaning a gradual transition from less to more (or visa verse) deprived areas. Whether this is a more realistic and intuitive mapping of deprivation I will leave to another occasion. This method obviously introduces fuzziness as the calculation assumes that a grid square that intersects LSOA does so with an equal area in each which is obviously not the case. The resulting map are shown below.

I have also gone up the scale from LSOA to Wards with IMD scores. As LSOA fall entirely inside the 624 wards no advantages/problems of naive smoothing. The maps are below;

The problem of blogs and research journals

2010-10-14T23:06:00.001+01:00

You may have noticed that I suddenly stopped posting my research without any explanation. It is not that I suddenly stopped working, I have been continuing to work quite hard, thank you. Its to do with the fact that over a glass of wine at one of these beginning of the new academic year mingle and be sociable events my supervisor gave me the task of writing a paper for a prestigious journal that peer reviews articles before acceptance. They not unreasonably expect original work that has not been published before. Now here is the problem. I "publish" rough and ready stuff on my blog. I had to and still having to decide what I can include in my blog that does not compromise the originality of my proposed paper. Of course the paper is a different style, it is more formal, more polished, more finished but it originates from the same brain, same data-sets, same journey. Where are the lines in the sand? Essentially it is my own integrity that is at stake here. I have to justify what I do to my self and I am sure that if persuade that judge who comes from a very puritanical background (thankfully tempered with a bit of compassion) I should be able to meet most peoples standards.

The real problem is one of timing and this where I think in this age where it is instantly possible to publish research that it is a tad unreasonable for journals to expect exclusivity unless they publish promptly. The fact is the process can be lengthy and it is not unheard of for a paper waiting two to three years before being published.

Research is a journey and my blog is a diary, yes a map of where I have been and where I am going. The argument is that I should continue but keep it to myself. I do not believe that is correct, I am publicly funded having the privilege of using data that in many respects is public, I feel that even though saying I have a duty to write a blog is overstating the case it has felt once I started that it is the right thing to do.

So my decision is I will continue to publish things as I do them and argue that the originality for journal publication is providing the rounded argument connecting different strands together in a formal and structured way. You certainly won't find any of that on this blog!!

CAD grid square analysis of violence and robbery in London

2010-09-29T09:37:00.001+01:00

These two maps use the same data as my other maps at Ward level but this time I have calculated the number incidents in a grid square that is 250 metres by 250 metres. This is the maximum accuracy of the Metropolitan Police Service Computer Aided Dispatch system which records these incidents. There are 26116 such squares in Greater London. This compares to 624 Wards

This is where scale comes into the analysis. Will analysis which is using units on average which are 42 times smaller and have a totally equal size (except those that clip the London Boundary) make a difference to the correlation and predictability between robbery and violence against person incidents?

The graph above shows the answer. There is a predictability of almost 0.6 and a correlation of 0.77. These are high figures.

Robbery incidents can predict violence incidents and visa versa

2010-09-28T11:53:00.001+01:00

This is the graph and map following the methodology of the previous post. The correlation is almost exactly the same as the robbery data arrays at 0.55 and 0.67 when the two West End outlier wards are removed. This raises the intriguing possibility that robbery incidents can predict violence incidents and visa versa.

Sure enough as the graph above shows a high level of predictability between the two incident categories. The correlations are 0.86 for all the Wards and actually goes down to 0.82 if the West End wards are removed. There are interesting aspects to this graph. Even though there is very high predictablity/correlation overall those seven Wards that have between 6 and 8 violence z scores appear to have a very low level of robbery predictability.

Is it possible that good policing may be an influence here?

2010-09-28T00:31:00.002+01:00

This a map of London showing the the levels of deprivation in each Ward based on the average score of the Lower Super Output Areas scores calculated for the English Indices of Deprivation 2007. As I have discussed in previous post criminology theories suggest that the higher the deprivation the higher the crime. So as I have carried out my violence analysis at Ward level I can investigate if there is correlation between my deprivation values for each Ward and my various incident occurrence values. First I am analysing robbery incidents.

The correlation between the two arrays of scores is 0.58 but goes up to 0.66 if the West End outliers are removed. More interesting is to plot the two set of values against each other to determine a linear regression line. This line has an equation that describes its slop and where it bisects the y axis. It also has an R squared value or coefficient of determination. This shows the degree of influence that deprivation has on the occurrence of robbery incidents (to put it simply). See here for a more detail. In this case it is 0.34 or just over third.

Now each Ward has a new value that can be measured above or below the line by using the equation to determine the score if it were on the line and subtracting that score from it actual score. This new score can be mapped to show those Wards that have a higher and lower score than predicted by the line.

We know from previous post that entertainment venues has a high influence on the occurrence of violence, including robbery, which explains some of the high scores. The Wards that do not have that "excuse" and have high scores are therefore interesting. So are the Wards with lower than predicted scores, especially those shown in green. Is it possible that good policing may be an influence here?

Google Earth London robbery map

2010-09-26T19:11:00.001+01:00

The information you get on this is my research literally as I am doing it so there is a quite a bit of experimental stuff. This is another example.

This is my latest development and an idea that I have not seen on web before and certainly not in the crime mapping environment. Instead of providing an interactive map I am going to provide a .kmz file which you can load into Google Earth and zoom in, zoom out, add your own features to etc.

You can get the file here at my new website - https://sites.google.com/site/saferview/google-maps

The top picture should be what you get when you load up the file. The legend for the colours can be found here in a recent post. You can zoom in and click on the wards to discover the numbers of robbery incidents that occurred in 2009. There are other numbers shown that you will understand if you have been reading my posts.

If you turn off the robbery layer by removing the tick you can see the layer of dots underneath. These are the CAD 250 metre by 250 metre grid squares (see previous posts , search on "square" or CAD in the search bar above). I have created a layer by doing a spatial join in ArcMap and counting the number of incidents in each grid square. The resulting layer creates a .kml file that is too large to display in Google Maps so I have only provided those grids squares with 26 or more robbery incidents or an average of one every two weeks or more. This gives a simpler and more focused display in my view. Yellow are the lower numbers, blue the higher and red in between. You can click on them to find the actually number which are shown next to "count".

By moving the transparency bar it is possible to see the layers together to find the robbery hotspot within the Wards. Then you can zoom in and see the area in which these robberies occur.

Violence classification including robbery mapped

2010-09-24T14:49:00.001+01:00

This my classification of violence at Ward level from police incident data in 2009. I have now included robbery data. Even though some incidents have codes that mean they could be counted twice in this classification I have decided to avoid this by using the following hierarchy.

1 and 29 - domestic violence all
1 but not 29 violence but not domestic violence
5 but not 1 and 29 robbery but not violence or domestic violence

The last two could possibly be changed so that robberies with the code 1 could be counted but it does not make a lot of difference as these are about 400 out of over 36,000 incidents (the combination of 5 and 29 is less than 100). My thinking that putting 5 and 1 together is tautology unless the incident had an additional violent element. This brings me onto an additional important point, we are not dealing with seriousness here we measuring numbers; a murder incident has the weight as a common assault.

The above is the table I based my classification on. I think this will be my last violence classification, for now.

Robbery map of London

2010-09-24T12:19:00.001+01:00

Back to violence. This is a map of robbery incidents. Let me explain in simple terms what a robbery is. It is a theft that involves the application of violence to an individual or individuals or threat of violence to achieve its purpose. Now because robbery, particularly street robbery is and has been for decades a police priority to reduce, only those where the final letter of the law is complied with find their way into the official crime statistics. Therefore you get pick pocket crimes that are simple thefts and bag snatches which most people would think of as a street robbery but in law is classed as a theft because the violence has been applied to the bag not the person carrying it. You can then get an assault followed by a theft and a theft followed by an assault which are not robberies. The use of a weapon to threaten potential violence is an armed robbery though there is no separator legal category for this in England and Wales.

But my data set of incidents is not quite so exacting as that, though it is normally based on the information given by the police officer initially investigating the case who would not have got through training school without learning the definition of robbery verbatim so it is likely to be broadly accurate.

Stop the Rot O'Connor

2010-09-23T16:45:00.003+01:00

I have been torn away from my analysis to study the headline news in the UK about the police and anti-social behaviour (ASB). Her Majesty's Inspectorate of Constabulary (HMIC) has published a number of linked reports about policing ASB in England and Wales which can be found here. They are important and interesting documents that unusually for the Home Office, are of varying quality. That aside there is so much I could comment on but I will be brief.

The authors behind this report are Sir Denis O'Connor and Prof. Martin Innes, the architects of Reassurance Policing. This makes everything a bit surreal because to large extent they are saying that RP and its offspring Neighbourhood Policing (NP) has failed to properly address ASB, probably their primary raison d'etre. In O'Connor's case he seems to acting as if this radical change in policing style which he championed and the Labour Government supported with vast amounts of money had not taken place in the last 10 years.

I argue that there are two styles of visible (uniform) policing - Community or Neighbourhood Policing and Response Policing. The two have to be properly balanced. Prior to O'Connor's NP initiative dating back to 2001 in Surrey and London and being adopted throughout England and Wales in the next four years to an extent that it is now embedded everywhere, the balance was out of kilter in favour Response Policing. Unfortunately the balance is now out of kilter the other way. Response times have been lengthened and the types of incidents police will attend promptly have been reduced to accommodate NP. What has the research published today found? ASB is best dealt with promptly and decisively as it is happening - the role of 24 hour Response policing. I am actually in favour of NP as it provides a balanced approach to policing but not at the expense of Response Policing. For NP to be effective the officers must command respect and be proactive using the full range of policing powers; and I would argue be on duty when the problems on their patch occur. Unfortunately at least two out of the three criteria do not apply to Community Support Officers, who are the backbone of NP.

Lastly, there is politics behind this. O'Connor I am sure pushed for the Police Confidence overarching performance target and the Policing Pledge under Labour as these support NP. The present Home Secretary stamped on them with her (in)famous shoes, and kicked them into the dustbin. This I think explains O'Connor's amnesia, he has to act like he is starting the fight for NP all over again with the looming government spending cuts. It also probably explains why the reports have an unfinished, unloved feel about them.

What is the difference between clustering and classification?

2010-09-22T23:26:00.006+01:00

This is my understanding of the answer to the question I posed. I have been doing a lot of clustering but in this post I presented my first classification. This is not my final version, I have more potential variables up my sleeve. The classification is a description of the variables based on their membership of a cluster. I now know why creators of geodemographic almost exclusively use the K means means of clustering; it gives an output of the where the final centres of each cluster is for each variable. This gives a pretty good idea of the nature of, in our case, the Wards in each cluster. This is the table for my classification;

The variables were adjusted around the mean, with 1 representing the mean value. This ensured that each variable was given equal weight in the clustering process. The description I gave to each cluster was based on the information in this table. For instance cluster 1 is high in all three variables so areas where violence is common, thus my description, cluster 4 seemed to me to be pubs and club venues from the very high value of the second variable.

I am not saying my descriptions are perfect, I am explaining the process. Of course the members of the clusters are varying distances from the cluster center (this information is also given) and therefore will not necessarily exactly match the mean characteristics of the cluster. It should also be borne in mind that it is the dominant relevant features of the Ward that are being described. It is highly unlikely that all parts of the Ward will comply with that description. We then get into a discussion about the scale of analysis which I have discussed before on this blog.

So what is the answer to the question? Classification is the next stage in the process after clusters have been created; classification involves the description of clusters based on the characteristics of the cluster members.

Domestic Violence incidents mapped

2010-09-22T13:59:00.003+01:00

The map shows the distribution of police incident domestic violence in London in 2009. I have included these data in my classification of violence in London. This was an interesting decision I include the data for a number of reasons. First is shown in the table below;

In my first post about the analysis of violence explained I would be using the three result code fields for incident, "1" is "violence against person". This is the basis of all this analysis. I am now looking at the combination of "1" with "29" - "Domestic Incident". The table shows the total number of incidents that contain these codes and the number of incidents that contain both. The combination of 1 with 29 is more than 4 times higher than 1 being recorded with any other incident code; it is therefore an important combination in the data set.

Second domestic incidents are a high priority for the Metropolitan Police with incidents of domestic violence a particularly high priority. Thirdly the data set for my clustering was biased towards non-residential locations as I used late night weekend incidents mostly associated with entertainment areas. Domestic violence by its nature mainly occurs in residential locations and thus counterbalances the previous bias.

There are arguments why I should not include DV incidents. The main one is that it may not properly record the occurrence of domestic violence as the more police treat it as a priority the more the demand will be for police to deal with it. So to a certain extent the demand is associated with positive police activity. This may be the arguments of police in the boroughs of Lewisham and Greenwich where domestic violence appears high. The fact is all demand is associated with police activity creating or maintaining confidence that police will deal with the incident appropriately. The point is the data are about the nature of the demand on police, but the above concerns are valid as domestic violence tends to happen in private and depends to a large extent on the victim reporting it to police whereas other types of violence tends to happen in public places and therefore is reported by witnesses and/or discovered by police.

Geopolisographics - an explanation

2010-09-22T11:21:00.002+01:00

In the last post I invented the word "geopolisographics" whether it will stick remains to be seen. It attempts to mimic the ancient Greek etymology of "geodemographics" which if broken down has the following meaning - "geo" meaning earth or world literally but is used to mean location, place, space; "demo" people; and "graphic" description. As the data I am using do not describe people directly such as age, gender, income, interests; but instead describes demands on police and police activity that is related to place and time I feel that replacing demo with poliso still makes the word pronounceable and better describes what I am doing. (My data of course indirectly describes people.) "Poliso" is derived from the ancient Greek word "polis" meaning city and "polissoos" meaning guardian of the city.

"Geopolisographics" - the analysis of police incidents by where and when they occur

2010-09-21T17:52:00.001+01:00

I am quite proud of the map above. It is my first classification map. It is in the style of geodemographic classifications but it is not geodemographics it is "geopolisographics" - the analysis of police incidents by where and when they occur. This is again based on "violence against person" police incidents in 2009. I have added domestic violence incidents into the mix. I will explain more later.

Mapping violent crime patterns

2010-09-19T21:58:00.001+01:00

This post is a continuation of the analysis presented in the last three posts.

In this post I am going to outline how I have clustered the Council Wards in London based on the "violence against person" incidents in 2009. I am clustering the demand profile that these incidents present to the police. The idea is to find similar Wards in London based on the variables I use.

The choice of variables is critical. I have hopefully shown that Wards can be grouped by number of relevant incidents in the year and whether these occur between midnight and 4am on Saturday and Sunday mornings or whether they do not occur at these times. I am therefore only using two variables at this stage.

This means they can be plotted in a two dimensional graph based on the incidents that occur at those two separate times. This is shown above.

I then load the simple three column spreadsheet into SPSS17 and perform two different types of clustering calculations. I have outlined in some detail how these calculation work in six posts starting here. I used two different methods - K means and Ward's Hierarchical (do not be confused Ward is the person who devised the method and nothing to do with Council Wards, its just a burden I have to bear when writing about my clustering analysis).

So this is what the map of London looks like when produced by ARCMap and the accompanying graphs in MSExcel.

I have specified six clusters and I have sized the graphs so the x and y axis have approximately the same scale. I have tried to keep the same colours in the maps and the graphs. If you look at the graphs closely you will see that the two different methods split the clusters up in similar but slightly different ways. What becomes obvious is that because the y axis has many more incidents in most cases that the clustering does not really take any notice of x axis. Therefore the maps are just clusters based on incidents that happen outside the Saturday and Sunday midnight to 4am. This is not good enough. I have decided that both variables are equally important as each other so I will have to make adjustments to ensure this is reflected in the clustering.

Mathematically this quite simple (I hope I have got this correct having made that bold statement). First I calculate the proportion or percentage of the x axis value is to the y axis value in each Ward. That is (x/y)*100. These percentages are then all added together and divided by the number of Wards to give an average percentage, which is in this case is 10.70% (to two decimal places). So to make the x axis variable have the same scale as the y axis; 100%/10.70% gives a figure of 9.34 (to two decimal places). This 9.34 is then used to multiply the x variable incidents for each Ward.

The resulting three column table is loaded into SPSS17 etc. and the following maps and graphs are produced.

The clustering now takes both variables equally into account but the two methods split the clusters differently especially the brown and orange. More in subsequent posts.

Identifying different London Ward violence profiles

2010-09-16T11:42:00.001+01:00

I have been analysing violence in London based on the demand profile for each council ward using police incidents that have been resulted as including "violence against person". I have discovered by this analysis that the period after midnight to 4am on Friday night Saturday morning and the same period Saturday night Sunday morning can be used to group wards. I am developing that idea further in this post.

London 2009 police "violence against person" incidents on Saturday and Sunday midnight to 4am showing council wards

The dark blue/purple show the most violent wards in London after midnight on Friday and Saturday nights. I have not checked each one but I think the blue/purple and red/purple cover most of London's major night drinking and entertainment venues.

London 2009 police "violence against person" incidents NOT on Saturday and Sunday midnight to 4am showing council wards

This map shows the violent areas of London for the rest of the week. Some wards appear purple on both charts but others change quite markedly.

London 2009 showing the percentage of police "violence against person" incidents on Saturday and Sunday midnight to 4am showing council wards

This map is slightly different. It is not based on the number of incidents, it is based on the proportion of incidents that occur in the early hours of Saturday and Sunday mornings. This allows the profile of less violent areas to be judged. It also justifies my decision to carry out MPS wide analysis to help interpret my Camden Borough data because it shows that the Gospel Oak one of only three wards (very light green) that have the lowest percentage of incidents that happen in the weekend period under analysis.

Next stage is to see if this data provides good clustering and classification material.

Temporal Profiles of Violence in the top Ten Wards in London

2010-09-15T09:43:00.002+01:00

These are again the top 10 violence wards in London in 2009 that I mapped in the previous post. This post shows the temporal profile of the occurance of these incidents. To me there are two distinct groups with one or two being in both groups. See if you agree with me.

Group 1 are those where most incidents happen in the early hours of the morning at weekends - Marylebone, St James, Fairfield and Romford Town. These are dominated by town centre entertainment venues. The other group is the rest, with Coldhabour being probably the best example, here the violence is spread more evenly over the week, so I suspect less dependent on pubs and clubs. Stratford and New Town and Shepherd's Bush show combination of both. It must remember that wards are relatively large units which can contain diversity of all types. Smaller geographical units of analysis may help with this.

Violence mapped by Ward

2010-09-14T16:53:00.000+01:00

London 2009 police incidents that included "violence against person"

This map shows the violent Wards in London according to police incident data. The Wards highlighted in dark blue/purple are shown in the table below.

Resilience Index and the Index of Multiple Deprivation

2010-09-13T14:28:00.001+01:00

In this post I am in danger of giving you more information that you will ever want or need about the Resilience Index (RI) and the Index of Multiple Deprivation (IMD) but here goes. The post is prompted by Matt Ashby's comment on the previous posting about the RI. My reply is relevant this post.

First a little table that shows the correlation between three indices, the RI, the IMD 2007 rank summary of Local Authorities (which can be downloaded here) and the rank index of IMD which is used as a data element within the RI - RIIMD. I have correlated all Local Authorities in England and London Local Authorities, called boroughs separately.

The first row shows a far higher internal correlation with the RI for England than for just London. The second row shows there is virtually no correlation between the RI and 2007 IMD for England but a negative (in practical terms positive as the indices are orientated inversely) correlation for London. The third column shows that the two ways in which the IMD has been interpreted at LA/borough level has no correlation for England but a strong correlation for London. What this shows is London is not typical of the rest of England.

The next table shows the rankings for London. The equivalent table for England is too large to include.

The Charts below show the above information in scatter graph regression charts.

Index of Resilence

2010-09-09T16:06:00.002+01:00

The picture above which looks like slate flooring is in fact a map of London Boroughs. It comes from an interesting new index commissioned by the BBC and complied by Experian which can be found in it interactive state with background documentation and spreadsheet here. The index ranks the 324 boroughs in England by how resilient they appear to be to Government spending cuts; 1 being the most resilient and 324 the least. It appears to me a wealth index in the broadest sense, that is, not just money. It is a sort of index of multiple non-deprivation. It is of interest to me because it is at borough level, an important operational level of policing and it allows me to compare my borough rankings of policing data with an index that broadly indicates underlying social issues that may affect the nature of the crime and incident demand on police. Unfortunately the multiple index does include some crime elements so it is not totally independent from my data.

Influences on violence

2010-09-08T23:35:00.000+01:00

Top 11 days of "violence against the person" incidents in London in 2009

Bottom 10 days of "violence against the person" incidents in London in 2009

Most police officers would agree, I think, that the biggest influence on violence is alcohol. I have used a Met Office website to look at weather conditions in 2009 that can be found here. Weather appears to influence violence through the mechanism of more excessive alcohol consumption, I think, when it is hot. This hypothesis is supported by the fact that the peak time for incidents is around midnight. Holidays such as New Year and hot bank holiday weekends that co-inside with half term seem add to the mixture. Halloween and extra hour at night due to changing the clocks also seems to affect things.

There appears to be a background level of violence of about 300 incidents a day (I am sure that 22/6/09 is a data blip due to the computer being out of action for about 16 hours) that is undeterred by anything including snow.

There is an interesting paper "The influence of weather on local geographical patterns of police calls for service." in Planning and Design 2009 Volume 36 Number 5. This looked at various weather factors and concluded that there was a relationship between weather temperature and disturbance incidents. The methodology looked at the temperature at the time of the incident. From my brief analysis I think that the influence of temperature continues even after it becomes cooler due to the alcohol factor.

Violence in London

2010-09-08T18:01:00.001+01:00

I am in the midst of analysing police incident data. I have decided to look at violent crime first.

Let me explain the features of the data set that are relevant to this post. An incident is created from information from a member of the public (or on occasions from police officer information). This is coded to the type of incident. There are about 100 codes that can be used in three fields. When the incident is completed the same codes are used in three fields to result or class the incident.

I am interested in just one of the codes "1" which is "violence against person". I am interested in this analysis of the demand profile so it would be logical to look at the type fields based on what the initial information is. I decided however to base my analysis on the class fields because I think this more accurately reflects the true nature of the incident.

So the incidents I have selected have the number "1" in one of the class fields. This gives me a rich data set of incidents to analyse, and one which I suspect is more reflective of events of violence than the recorded crime data set.

Thumbs up for Safer Milton Keynes

2010-09-07T12:54:00.000+01:00

This crime map is part of the Safer Milton Keynes website. It is the best I have seen of the UK type of crime map. It uses Ordinance Survey rather than Google Maps. What I like about it is the way it has good background information about data sources, limitations of the data and the analysis. It integrates council antisocial behaviour with police crime (from the crime reporting system I am assuming).

There are a number of non-standard features about this map though. It is based on Milton Keynes Neighbourhoods and Estates, which as far as I can make out are definitely not co-terminus with Lower Super Output Areas and perhaps not Wards, if grouped; the crime categories are not totally the same as Home Office ones; and I suspect the anti-social behavior categories are also devised locally. I am not going to be critical of this because the categorisations seem better to me and more understandable to the public.

I wonder if they are considering including police incident data.

Shedding more light on the Dark Figure

2010-09-03T08:29:00.000+01:00

I will be giving a presentation based on the abstract below at the Crime Survey Users Meeting in London in December.

By the use of victimization survey methodology the British Crime Survey attempts to measure the numbers of different types crimes that have occurred to individuals and households. The statistics produced allow the percentage of crimes that were not reported to and recorded by the police to be assessed.

The presentation will discuss how the British Crime Survey could shed more light on this dark figure. The relevant section in “Crime in England and Wales 2009/10” is 2.5 Reporting Crime.

The discussion will include the following topics;

• Is the methodology for calculating the dark figure correct?
• Should the questions address why people report crime to the police not why they do not?
• Is the dark figure a measure of the level of confidence individuals, groups, communities has in the police and the criminal justice system?

Murder Maps - the best risk assessment aid

2010-09-02T12:43:00.000+01:00

I am increasingly liking murder maps. They are simple, straight forward and easy to interpret. They are the quickest way to understand where the safer and less safe neighbourhoods are in major towns. Something that the people who live there probably know but essential risk assessment information for the intrepid traveller.

The first map shows murders in Washington DC, USA 2005-9, the link is here. The designer has helpfully shown which area is included, which means that it is obvious that there are almost no murders west of the Potomac River and many to the east

No murder housing near the Cathedral west of the Potomac River

Many murders housing east of the Potomac River

Its not cricket

2010-08-30T17:53:00.002+01:00

We are told that Diana death was accidental, Dr David Kelly's death was suicide and Bob Woolmer's death was by natural causes but many people believe that one or more of these deaths was murder.

Bob Woolmer, for those who do not know, died on 18th March 2007 in his hotel room in Kingston, Jamaica when he was the Pakistan's cricket coach at the Cricket World Cup in the West Indies. His death was investigated as murder with suggestions that it was in some way linked to Asian betting syndicates. Perhaps he had found out that some Pakistan players colluding with the syndicates contributed to Pakistan losing to Ireland the previous day and was going to spill the beans.

I had been following Bob Woolmer's career since the early 1970s because I had been told that he was an old boy from a school I went to in Derbyshire. It appears that this "fact" is actually wrong but it has not meant I did not believe it for about 40 years.

And this is my point - a person's perception is often more relevant and powerful than facts. Fear is based on perception.

The British endearingly often fall into the trap of thinking that because a person speaks English and perhaps plays cricket that they have the same background, attitudes and values as them. The fact is Pakistan is about as different as you get to Britain; crime and the rule of law are a world apart.

Everyone in England seems to assume that the latest cricket betting scandal, this time involving Pakistan, is about individual cricketers making money. What if money is not the primary motivation but fear? Are they seriously advocating banning an 18 year old (probably the best fast bowler of his age in the history of the game) for life because he was probably doing what he was told and feared the consequences if he did not?

The historical exploits of James Cousins

2010-08-26T17:39:00.000+01:00

There is something very British about this story and one that is not totally unrelated to the previous blog posting.

James Cousins, an elected Conservative Councillor seems to have spent a disproportional amount of time over the last 18 months point mapping robberies and burglaries on Google Maps for the benefit (primarily) of Wandswoth residents. He had been regularly given the information I assume from the Metropolitan Police Service (MPS) in his official capacity as part of the Wandsworth Community Safety Partnership.

James Cousins obviously thought that the information he was being given was far more useful than the official MPS website and something which he should share. By producing point data maps with details of the crime for the public he was, out of ignorance or through bravadory going against MPS policy and current Data Commissioner's advice regarding privacy. But he managed to do it for 18 months without problem or complaint from those supposedly affected. He has only had to stop because someone in Harrow Council did not like what he was doing. The reasons given were privacy, raising the fear of crime and giving criminals information that they might fine useful.

James Cousins like a batman given out wrongly in cricket has walked back to the pavillion without a hint of dissent. I hope the history of crime mapping for the public will remember his exploits.

At last a "crime" to investigate!

2010-08-25T11:43:00.002+01:00

Colin D from SpotCrime kindly alerted me to the fact that SpotCrime and CrimeReports are currently in dispute. He gave me this link to a commentary on the case entitled "Who owns public crime data?". A very important and relevant question and one which the the article mysteriously fails to address let alone answer. It does however, very usefully have the full 52 page civil court complaint made by (effectively) CrimeReports against SpotCrime.

The allegation, put simply, is that SpotCrime is taking data in bulk from the CrimeReports websites by sophisticated means which SpotCrime is having difficulty preventing. The data that they are taking has been provided to CrimeReports by numerous police departments across the country who pay CrimeReports to provide the data (relating to police incidents, including crime) to the public in an approved format and detail. SpotCrime admit (it appears) to scraping the data from the CrimeReports website and then putting it on their own websites which unlike CrimeReports includes advertising to generate income. They also are selling the data in the form of alerts to media companies it appears. SpotCrime defence seems to be that the data are public property, in the public domain and therefore they have as much right to it as CrimeReports.

I have been grappling with the problem of who owns police crime-related data for about 20 years now. In 1994 I wrote "Facilitating Public Access to PNC stolen vehicle data" for the Association of Chief Police Officers and for the Home Office. This was the first crime data that the police in the UK provided to the public on an enquiry basis and it used a privately owned commercial company to do so who effectively sold the information to the public. The report, which I am proud of, had to cover ownership issues and whether the data had monetary value; it also covered the purpose of providing it to the public. At that time there were not issues regarding the public nature of data though the Freedom of Information Act subsequently enacted has changed this.

I am not going to go into the details of my argument because it would take too much space but it is clear to me that despite the public nature of the data they are owned by the police departments that produced them. It becomes jointly owned by CrimeReports who have entered into a mutually beneficial contract. CrimeReports have a degree of ownership due to adding value to the data that carries with it a duty to protect its integrity and security.

Having come to this conclusion, that carries with it the assessment that the data have monetary value, I see CrimeReport/Spotcrime case, despite it not being treated as such, as an allegation of theft, though it is probably easier to prove computer hacking offences.

Good & not so good Crime/Policing Mapping Sites 4 – CrimeReports

2010-08-24T15:34:00.002+01:00

To me CrimeReports is an impressive site - thumbs up, but a small thumbs up because it is not without its faults.

CrimeReports go about things in the right way, they obtain their data from police agencies, so my confidence in the provenance of their data is high. They make their money from the data provider so there is no annoying advertising on their site.

CrimeReports philosophy seems to be that people should have access to information about crimes that are going on around where they live, work and frequent. It is therefore best viewed on a small neighbourhood scale.

What are the faults?

The main one is to do with the title - Crime Reports. This is an ambiguous title and one which makes most people assume (I think) that the data come from the police crime recording system that has a comprehensive record of reported crime. This appears not to be case. Most, if not all the data comes from the police Computer Aided Dispatch (CAD) system that manages the deployment of police units to incidents. The problem with this is that a significant proportion of crimes are reported without incidents being created and therefore the data is not as comprehensive as the title suggests. Secondly I suspect that CAD system in the USA, like the UK, is not standardised so it is difficult to compare different police force areas on the basis of this data.

This is not huge difficulty and one of semantics which can easily be solved by CrimeReports being bit more transparent.

After all it would be churlish of me to be over critical as I am advocating Policing Maps for the public based on incident data.

The thin blue line

2010-08-24T11:38:00.000+01:00

If you want a bit of a laugh and have 10 minutes to spare this is for you. It is of course not based on reality!!

Crime-ridden Finland - I do not think so.

2010-08-23T23:15:00.000+01:00

The last blog about Mexico reminded me how misleading United Nations crime figures of countries are. The table above shows the top twenty countries in the world ( plus Mexico at 39) regarding recorded crime per capita. This has Finland as the third most crime ridden country in the world with an apparent crime rate eight times higher than Mexico. The reason why I pick out Finland is because it was nominated by Newsweek magazine as the best place in the world to live and the country with the fourth best quality of life.

Clearly a high recorded crime rate can be an indication of a well run efficient society with probably relatively low crime. Equally a low recorded crime rate can indicate inefficient, badly run, high crime societies. There are so many counterbalancing forces here that in fact the total recorded crime rate actually indicates very little for comparision purposes between countries.

Fear, Fear and more Fear

2010-08-23T14:47:00.001+01:00

This blog has a strap line "crime, fear and mapping"; these maps probably contain the most fear and crime that I have seen. There is a terrible turf war going on in Mexico over drugs, mostly to do with the battle over who is going to control the the export market to the USA. The top map is recent and the lower one dates back to just before the leader of the Beltran Levya gang was killed in a shotout against at least 200 army personal around Christmas 2009.

I post the maps today because of this story. You can learn more about the ongoing saga of murder and mayhem by following the links on the BBC website. This is the one I found most distressing and this the most shocking.

Good & not so good Crime/Policing Mapping Sites 3 - SpotCrime

2010-08-20T23:10:00.001+01:00

I have tried my best to like SpotCrime because I think the people behind it hearts are in the right place and because it covers large areas of USA and parts of the world. But the lack of information about where specific data have come from and the non-standardisation of data between different places makes my confidence in the accuracy, comprehensiveness and the appropriateness of the maps very low. They are first generation crime maps for the public and of the not so good sort. The quality control is not there. There may be some quite good individual cities but they are more than let down by the bad ones.

The software supporting the site is good though. SpotCrime seem to be developing mylocalcrime.com. They claim it is fast and they are right - impressively so. The software behind reporting a crime is good. The way in which you specifically locate the crime on GoogleMaps to give highly accurate co-ordinates is something all police forces should duplicate.

Good & not so good Crime/Policing Mapping Sites 2 - IR3

2010-08-20T11:24:00.002+01:00

These are images from the L and A Consultants webpage. It appears that Leicestershire Police have an impressive integrated police activity mapping and analysis system (if it works as well as the publicity blurb). For that they get a thumbs up. It appears to be only available to police and not the public. Therefore it also gets a thumbs down.

Now if they or any other force who is using the system did publish it on the web for the public it would be ground breaking and what I would happily define it as second generation.

Good & not so good Crime/Policing mapping sites 1 - MyNeighbourhoodupdate.net

2010-08-19T14:04:00.000+01:00

I try to keep up to date with what is out there on the Internet relating to crime/police mapping so I thought I would start a series of what I think is good and not so good. Please feel free to comment (in English) and perhaps even suggest appropriate sights for me to look at, the newer the better.

The first site appears relatively new and approaches what I see as second generation crime mapping. To put it simply first generation by my definition maps the location and times of crimes to a varying degree of accuracy and comprehensiveness. Second generation crime maps should focus much more on police activity tackling the policing problems within neighbourhoods with the purpose of addressing communication, reassurance, crime prevention, confidence, performance and funding issues. Second generation crime maps are therefore "Policing Maps" for the public.

I give Myneighbourhoodupdate.net a big thumbs up because it uses police incident data to map police activity.

Explaining clustering to non- mathematicians and non-geographers 6

2010-08-11T19:33:00.001+01:00

The above shows the result of the Ward Hierarchical Cluster method. It is exactly the same as the Centroid Hierarchical Method except it uses the squared Euclidean distance as a measure of variance between clusters and within clusters. At every stage it forms a cluster of forces whose distances apart have the minimum deviation from the mean (average) distance from a centroid that could be created at that stage and the maximum deviation from any other centroid that is already present or could be created at that stage. It uses a calculation known as Sum of Squares. The formula can be found here and an explanation of how it fits into the logic of statistics, such as standard deviation here

K-means is another clustering method. It is not hierarchical which gives it less rigidity and more flexibility. It uses the same measure as the Ward method but its process is different. The users have to nominate how many clusters they want to be formed. The computer then puts that number of centroids, in our case in the 28 dimensional space, randomly (though in SPSS 17 it is always the same places as the results are always the same if the order of the forces is the same). The computer, through a number of tries or iterations, attempts to achieve the Sum of Squares minimisation within clusters and maximisation between clusters discussed above until it can do no better. You can see the non-hierarchical flexibility with Cleveland (in the North East) starting purple, going green, then going back to purple; something that is not possible using a hierarchical method.

Explaining clustering to non- mathematicians and non-geographers 5

2010-08-11T10:57:00.002+01:00

Just to continue with the visualisation of the centroid hierarchical clustering process - this is a dendrogram (all these new words I am learning!). It has taken me a bit of time to read what it is saying but I think I have got it. The reason why it is shown vertical as standard is because I think it should be read from the top left. If you remember the first merging of clusters (stage 1) was 2 & 33. Then it shows all the forces that are initially clustered with that cluster which becomes cluster 2. It becomes a bit complicated but it does logically tie in with the Agglomeration Schedule shown partially in the previous post. I have not gone into too much detail because practically it should be read from the right to the left, that is, as if the process was divisive.

If you now read this in conjunction with the maps and diagrams in the 3rd part of this series it all becomes quite straight forward.

It is actually very useful as it allows me to quickly see the similarities and variations produced by the different clustering methods.

Explaining clustering to non- mathematicians and non-geographers 4

2010-08-10T10:32:00.004+01:00

In the previous posts on this topic I have given details of my dataset of 28 variables of police force expenditure, discussed how this can be mapped in 28 dimensional space to provide unique locations for each force, and then introduced the concept of dividing the forces into hierarchical clusters based on the locations of the forces. The process I illustrated was in the previous post was the divisive or top down approach that starts off with all the forces in one cluster and then divides into two, then three, then four clusters, etc. until there are 43 clusters containing one force each. This is how most non-mathematicians approach clustering I think.

An alternative approach and one which is simpler mathematically is the bottom up or agglomeritive approach. This starts with 43 separate clusters and ends up with one through the merging of clusters.

The important thing to grasp in this post is that the criteria for clustering is based on a measurement of distances between clusters and this distance can be measured in various different ways. The example I am illustrating is calculated using SPSS version 17 software. I have selected the euclidean distance measure (squared - recommended by SPSS - gives longer distances more weight), which is basically the shortest distance between two points and measuring to the centre (or centroid) of the cluster. The process uses the agglomeritive approach.

The process starts with 43 clusters each with the membership of one force. The centroid of each cluster is therefore the location of that force in the 28 dimensional space that the computer calculates by plotting the values of the 28 variables relating to each force. The computer is then asked to find the closest two centroids (euclidean distance wise) which happens to be Bedfordshire and South Yorkshire (these are the most similar forces as far as expenditure patterns are concerned). It then merges the two clusters together (now there are 42 clusters) and calculates the centroid of that new cluster. It then looks for the closest two clusters again. This time Derbyshire and Kent clusters are merged and the centroid calculated of that new cluster (now 41clusters). The closest two clusters are again found. This time the Bedfordshire, South Yorkshire cluster is merged with Durham (40 clusters).

1 Avon & Somerset, 2 Bedfordshire, 3 Cambridgeshire, 4 Cheshire, 5 City of London, 6 Cleveland, 7 Cumbria, 8 Derbyshire, 9 Devon & Cornwall, 10 Dorset, 11 Durham, 12 Dyfed-Powys, 13 Essex, 14 Gloucestershire, 15 Greater Manchester, 16 Gwent, 17 Hampshire, 18 Hertfordshire, 19 Humberside, 20 Kent, 21 Lancashire, 22 Leicestershire, 23 Lincolnshire, 24 Merseyside, 25 Metropolitan Police, 26 Norfolk, 27 North Wales, 28 North Yorkshire, 29 Northamptonshire, 30 Northumbria, 31 Nottinghamshire, 32 South Wales, 33 South Yorkshire, 34 Staffordshire, 35 Suffolk, 36 Surrey, 37 Sussex, 38 Thames Valley, 39 Warwickshire, 40 West Mercia, 41 West Midlands, 42 West Yorkshire, 43 Wiltshire

This Agglomeration Table is produced by SPSS. It takes little bit of understanding. I listed the forces in alphabetical order so the numbers relating to the clusters relate to the forces as shown (but remember by cluster 2 by stage 3 has two forces in it 2 & 33, you are given help in this in the last 3 columns). The stages refer to the stage in the process, so stage 1 is when 42 clusters are formed from 43. The Coefficient column gives an indication of how good the fit of the clustering is. For instance it appears that 33 & 32 clusters (stages 10 & 11) are a better fit than 34 clusters (stage 9).

I am interested in the Metropolitan Police - 25 so it the other end of the table that is of interest.

Cluster 3 (Cambridgeshire) is merged with cluster 37 (Sussex) at stage 33 (11 clusters). At stage 34 the Metropolitan Police (25) is merged with that 3 cluster. This means that the Metropolitan Police is one of the last forces to be put in a cluster with other forces but it is by no means the most dissimilar force as far as expenditure is concerned. It probably comes about 7th in the list behind Cumbria, City of London, Avon and Somerset, Warwickshire, North Wales and Norfolk. Interestingly 6 clusters is a better fit than 7 clusters.

The maps of clusters 2 to 7 are displayed in the previous post. Even though the actual process is agglomeritive it is easier for non-mathematicians to visualise it as if it is divisive.

Explaining clustering to non- mathematicians and non-geographers 3

2010-08-06T19:55:00.000+01:00

In the last post I explained how the expenditure patterns of forces can be mapped using 28 variables to create exact locations for each force in a 28 dimensional space. In this post I provide hopefully self-explanatory maps and diagrams of the results of one of the clustering methods. The diagrams of the dots, are of course, purely illustrative as I, nor anyone else can depict 28 dimensions in two dimensions. This is a hierarchical method of clustering based on centroids. I will explain the method in the next post.

What I want you to note now is the fact that hierarchical clustering starts with one cluster which is divided into two, the third cluster involves the division of one of the two clusters, four clusters are made through dividing one of the three clusters, etc., etc. For instance, the two clusters shown above consist of Cumbria (in the northwest) and City of London Police (in the centre of London) as one cluster and the other 41 forces in the other cluster. The third cluster is made from dividing the 41 forces. The fourth by dividing Cumbria and the City of London, though if the data were different it could be the one of the offspring clusters of the 41 that was further sub-divided at this stage. This family tree type structure with parents and offspring - hierarchical provides a rigid structure which only allows the division of existing clusters. This means that after the first division Cumbria and the City of London could never be clustered with another force. So if the "best fit" (I will try to explain this later) at the seven cluster stage involved Cumbria (say) being clustered with other forces this will never be achieved using this method.

Explaining clustering to non- mathematicians and non-geographers 2

2010-08-05T17:41:00.001+01:00

What I am trying to achieve with clustering is to group police forces together based on the similarity and differences in the data I have selected from the HMIC value for money spreadsheet as I have discussed in the previous post.

Importantly each datum relates to a police force (via the row in the spreadsheet) and to specified expenditure (via the column); and each datum has a numerical value.

Now this is what you must grasp - that numerical value (in this case all percentages) becomes a co-ordinate in a similar way to map co-ordinate. On a flat two dimensional map a location can be described by a two coordinates, X,Y; latitude, longitude; etc. By selecting any two of my 28 variables the locations forces can be plotted on a flat "map". Those forces whose variable values are close numerically are close on the map, those that are more different numerically are further away on the map. If three variables are selected the map becomes three dimensional. If I include all 28 variables then the map becomes 28 dimensional. This is of course impossible for a human to visualise but easy for a modern computer to chart, but of course not display.

The fact that you must accept is that each force is very precisely located in this 28 dimensioned map. Each force has one location and this point will always be the same if the same variables are used with the same values. It does not matter if more forces are added or taken away, this will not vary the location of the other forces. This mapping of the forces in imaginary space is the first part of the clustering process. You will not become confused later in the process if you tell yourself that the locations are fixed and do not move for the rest of the process.

So the locations of the forces are now fixed for this calculation. Now the next part of the process is where the variations in the end result is introduced. How do you decide how to cluster the above set of dots into 2, 3, 4, 5, 6, 7, etc groups? That is for next time.

Explaining clustering to non- mathematicians and non-geographers 1

2010-08-04T23:01:00.002+01:00

This is really my project for the summer - learning and understanding statistical clustering, classification, and ranking of multiple variables based on location.

I am going to attempt to explain why I am doing this and how in a non-mathematical way. If you want it explained in numbers and formulae I recommend my hero in these matters Dr Kardi Teknomo who gently guides you through the subject in his brilliant tutorials which can be found here.

I have drawn your attenion to HMIC reports and an Audit Commission Report in recent posts. These compare different aspects of Home Office police forces in England and Wales' performance and expenditure. As these police forces have unique geographic juridiction these comparisons are geographically based. I am using data from the HMIC police force value for money profiles that can be found here. And specifically from the spreadsheet that can be found here. My interest is specifically to do with the Metropolitan Police Service (MPS) with responsiblity for London. I had a quick look through the report. The first page has the following;

My impression of the data presented was that the MPS is nothing like the forces it is grouped with. So I thought I would try group the most similar forces together using data from the spreadsheets and standard clustering techniques and at the sametime learn how to apply the most useful of them to my data from Camden and the rest of London.

If you have been following my blog you will know that I do not like comparisions that are standardised using "per head of resident population". It is particularly unfair on London and other places whose actual population is greatly swelled by those working, shopping, being entertained and holidaying there. A compromise is made for the City of London by calculating their figures on a population of 316,500 where in fact the resident population is about 8,000, but for nowhere else. So I disagree with this statement shown on the first page of charts;

I have therefore steered away from using data standardised in this way. What I have used are data that show how expediture is allocated as a percentage of another figure. These proportion figures overcome the problem that the MPS is so much bigger than any other force. The data used are "non-staff costs as % of staff cost"- columns AX to BE and "Supplies and Services as % of staff costs" - columns BQ to CA on the first sheet of the spreadsheet. And "% of the total Police Officer and PCSO workforce by rank" - columns AD to AN on the second sheet. This amounted 28 variables to be used in the clustering process.

I reason that how a police force chooses to spent the money allocated to it provides an indication of similarity and differences of how the force is managed, structured and its priorities. I am interested in which forces, on this basis, are most similar to the MPS and generally the forces that are most different to the others.

I will explain how this is done with reference to the maps above and others in Part 2.

The Audit Commission Report

2010-08-03T11:11:00.002+01:00

The Audit Commission published a report into policing in July 2010 (post-election) based on research towards the end of 2009 (pre-election) entitled "Sustaining value for money in the Police Service". It is a report by accountants for accountants identifying where cost savings can be made. In places I think its analysis is a little too simplistic, for instance regarding the relationship between confidence in the police and expenditure. I did find this visualisation of the apparent mismatch between demand and resources interesting, though I would like to see the raw figures to understand what is and is not included.

A pre-emptive strike to defend Neighbourhood Policing?

2010-08-03T09:40:00.000+01:00

In one of my recent blogs I referred to "Valuing Policing. Policing in the Age of Austerity ". This is a report by Her Majesties Inspector of Constabularies (HMIC) into the future of policing in the climate of budget cuts. I have read it now. Here are my very brief comments.

Its a good, intelligent and readable report, but to understand it properly you have to grasp the motivation of the person behind the report, Sir Denis O'Connor. He, as I mentioned before, is the champion of Neighbourhood Policing (NP). He therefore wants to ensure that this radical change in policing style is not ditched by the government and forces in the budget cuts. This report can therefore be seen as a pre-emptive strike.

He cleverly includes NP with Response Policing (RP) in his statistics - two different policing styles in my opinion - as complimentary core frontline policing activities. Not as competing styles for funds. The chart I show above is an example of that. The fact is the reason why there are more police available for patrol on Monday morning than Friday night is because NP officers rarely, if ever work nights. A situation that NP needs to address if it is to survive and flourish.

Symbols of Fear

2010-08-02T13:33:00.001+01:00

An artists impression of the proposed new US Embassy in London

I have been reading a book edited by Michael Sorkin entitled "Indefensible Space. The Architecture of the National Insecurity State". The title draws on Oscar Newman's Defensible Space Theory. Within the book there is a chapter by Trevor Boddy - "Architecture Emblematic. Hardened Sites and Softened Symbols." Always on the look out for symbols of fear and insecurity I found this quote interesting;

"There is one building type - because of prominence, vulnerability and political associations - that has been the subject of hardening for far longer than others and may predict where the architecture of the insecurity state is headed. The building type is, of course, American embassies - that most emblematic of all emblematic architectures."

Nice to see that the architect is trying to blend in with the existing buildings like the Tower of London about one mile downstream and The Keep in Windsor Castle about 22 miles upstream.

Good to see that there is going to be some water in the new building's moat. This has been superfluous to requirements for a few hundred years in the moats around the other two buildings.

What are embassies for anyway? This article discusses this question.

Towards Second Generation Crime Maps for the Public - the paper I presented today.

2010-07-28T17:33:00.003+01:00

This paper briefly discuses the history and possible future of crime mapping for the public in the England and Wales with reference the politics, policy and policing decisions that have aided and at times impeded implementation. It fits in well with the theme of this conference, Map Asia 2010; “Connecting Government and Citizen through Ubiquitous GIS”.

First the politics: crime maps for the public was a Conservative Party policy initiative that became an election manifesto promise in Boris Johnson’s London Mayor electoral campaign in May 2008 (Conservative Party 2008). This made it clear that the politician’s view was that the purpose of crime maps is to provide police performance accountability to the public at a local level. Boris Johnson became Mayor of London on 2nd May 2008 and immediately required the Metropolitan Police Service (MPS), which polices London, to provide crime maps to the public through the Internet. Figure 1 shows the website that was created by the MPS within weeks.

This prompted Labour Party Home Secretary to make it a requirement for all Home Office police forces in England and Wales to follow London’s lead. The National Police Improvement Agency (NPIA) became involved in providing a national solution that was launched in October 2009 - http://www.crimemapper.co.uk/.

It is obvious from the Conservative Party publication that they were expecting police to produce point data crime maps similar to those seen at http://CrimeReports.com. Legal policy issues pertaining to privacy that were raised by the Data Protection Act Commissioner means that the greatest resolution visualised are choropleth polygons at Lower Super Output Area level (1) . Assistant Chief Constable Steve Mortimore head of the NPIA crime mapping unit speaking at the National Crime Mapping Conference on 10th June 2010 was confident that the arguments regarding privacy were moving in a direction that will allow police to publish point data for all but the most sensitive types of crimes. This sends the policy ball to the police side of the net: is the police data of sufficient accuracy and quality to allow public scrutiny? The problem is not that the data does not have geocoded attributes it is the fact that it is either postcode address or grid square based both of which provides problems with point accuracy. The MPS crime recording system provides one metre squared point accuracy using the national co-ordinate system. This is fine when the location is a residential premise with a small land area. Problems arise in public spaces and large premises where the location of the crime may be shown all to one point such as a road junction or an administration office based on the postcode address even though the crime may have occurred tens or even hundreds of meters away. The MPS (3)) incident recording system designed almost 30 years ago when computer memory was at a premium uses centroids of grid squares of 250 metres by 250 metres to locate incidents. Even though this is technically point data it cannot be regarded as accurately pinpointing the incident.

Another policy issue was tied up with copyright and licensing. It looked at one stage that the British Ordinance Survey (OS) could derail the crime mapping for the public initiative (Guardian 2008) based on its insistence on enforcing its rights. It seems that the OS overplayed its hand. In November 2008, after advice from Sir Tim Berners-Lee, the founder of the Internet, Gordon Brown, the then Prime Minister announced that OS had until April 2010 to release its mapping data for free re-use. This data can now be obtained from http://www.ordnancesurvey.co.uk/.

What the politicians overlooked was the fact that police recorded crime data is an extremely complicated dataset which when applied to a local level is very difficult for even an expert to accurately interpret. The dataset is incomplete due to: the under-reporting and under-recording of crime; the counting rules which though logical are intricate; and the crime classifications which are only fathomable to those with detailed knowledge of criminal law statutes. The mere geovisualzation of the data does not simplify matters, it may further complicate by adding the dimension of the incompleteness and potential inaccuracy of location to the mix.

Another fact that the politicians conveniently overlooked was the fact that crime statistics are not trusted by the public as representing reality (UK Statistics Authority 2010). In 2001 police identified that even though official statistics were showing a reduction in crime the public’s fear of crime was rising. This mismatch was termed the Reassurance Gap (RG). The cause was attributed to the police activity over the previous decade that had pursued crime reduction targets to the detriment of making the public feeling safe and secure in their communities (Povey 2001). Radical changes in policing style were introduced because it was feared that the RG was to continue to widen confidence in police would decrease leading to a reduction in co-operation with police and compliance with the law (Hough 2007). These changes involved re-engaging with the public through the introduction of dedicated uniform foot patrol teams to each council ward(2) in England and Wales. This proactive, community based, problem solving, high visibility Reassurance Policing style had its theoretical justification in the crime theories of incivility originating from New Jersey, New York and Chicago (e.g. Wilson 1975, Garofalo and Laub 1978, Hunter 1978, Wilson and Kelling 1982 and Skogan 1990 reviewed in Taylor 1999) on the other side of the Atlantic and the Signal Crime Perspective from Surrey and London (Innes 2004).

The Incivility Theories and the Signal Crime Perspective emphasise the way in which incidents and crimes shape people’s fear of crime in their community. It is the incivility or in other words antisocial behaviour such as littering, graffiti, begging, noise and intimidating groups of youths that are all pervasive and unavoidable that is often of greater influence than the rare serious crime incident. This negative communication, so the theories assert can be mitigated by reassuring police activity that tackles the issues that are of most concern to the public. In this way fear of crime can reach a level that is in line with the risk victimisation.

The first generation of crime maps for the public have three major failings;

• they exclusively use police recorded crime statistics,

• they do not communicate police reassuring police activity, and

• they provide scant information about individual risk of victimisation.

Reassurance Policing is the now the primary policing strategy (though the name has disappeared in favour of the Neighbourhood Policing) with the introduction of the over-arching police performance indicator (PI) in late 2008 of increasing the public’s confidence in the police. Whether this PI and strategy will survive the change in government is yet to be seen. What will definitely survive and flourish, with the Conservative Party as the dominant partner in the coalition government, are crime maps to the public as a means of providing police transparency and local accountability (Herbert 2010).

Research is being undertaken by the author to link police activity to the public confidence PI through geographical analysis using data from Metropolitan Police Service (MPS) information systems. A product of this research is identifying how police activity can be mapped and scientifically compared at different locations. The methodology being used provides ample information about how second generation crime maps for the public can be created that address the deficiencies of the first generation maps.

The methodology involves populating five domains with relevant data. These domains are;

• Police activity

• Public demand on Police

• Characteristics of victims and offenders

• Characteristics of people and place

• Public attitudinal survey regard policing and crime

The links between first and last domains can be isolated and analysed by understanding variations in the other domains over space and time. Variations in the occurrence of different types of crime can be explained in terms of the different combinations of likely offenders and likely victims at likely locations and at likely times. This is illustrated in figure 2 below that is the part of a pilot study for the research in the London Borough of Camden using data from the characteristics of victim and offender domain which are drawn from the police crime recording system.

The risk of being a victim or an offender of crimes varies with gender, age and lifestyle. A variety of second generation crime maps for the public need to allow risk assessments that are crime type, location, time, gender and lifestyle specific. This information should provide a realistic level of the fear of crime leading to appropriate crime prevention actions, reduction in crime and increased confidence in the police.

Understanding of variations of the characteristics of different places allows crime patterns to be more intelligently analysed. For instance, the “pickpocket person” map in figure 2 shows hotspots that co-inside almost exclusively with areas of restaurants, public houses and other entertainment venues (robbery person occurrence is a little more diffuse). Lifestyle in the context of this research not only refers to whether or not people go to restaurants and public houses but other factors such as income, type of housing and employment. Geodemographic classifications (4) and the English Indices of Multiple Deprivation (IMD) 2007 (Noble et al 2008) (5) are used in the characteristics of people and place domain. Regression analysis of the homes of victims of crime and the IMD has been carried out in the London Borough of Camden. This has found that people that live in areas of higher deprivation have a much higher risk of victimisation of violent crime (that is recorded by police) than areas of lower deprivation, even when these areas border each other. This points to lifestyle associated with income.

Figure 3 shows the analysis of the interaction of the public demand and police activity domains. It can be inferred that people living in the grid square highlighted have less confidence in the police than people living in other grid squares where the response rate to emergency calls is faster and where police have tackled the underlying problems causing the calls to be made. The publishing of such maps to the public of incidents rather than just recorded crime gives a more comprehensive picture of the crime and disorder problem at locations and makes police accountable for addressing them fairly and effectively. In the England and Wales police act in partnership with borough councils to address local crime and disorder problems. Consequently the Councils collect valuable data regarding antisocial behaviour such as littering, noise and graffiti; and also logs of CCTV observations. This data gives further insight into the characteristics of place that gels with incivility theories and which is pertinent to analysis of police demand and activity.

The interpretation of police activity data is complex. One of the best sources of such data is Global Positioning System (GPS) signal data transmitted from marked police vehicle’s mobile data terminals

Figure 4 shows an example of those transmissions mapped first within ARCGIS, the layer converted into a KML file and then mapped on Google Earth. Foot patrol Officers’ personal radios transmit GPS signals but these are not stored at present and therefore not available for analysis. These GPS signals provide intriguing possibilities for the second generation of crime maps. Reassurance Policing and its protégée Neighbourhood Policing is about police forces demonstrating visibility, accessibility and familiarity. This resulted in an emphasis on high visibility foot patrols. Is it going a step too far to include live feeds of police patrols to crime maps for the public? A website showing the position of London Underground trains in real time demonstrates how this could be done (6).

Finally the attitudinal data domain: the British Crime Survey provides the data on a force wide basis for the overarching performance indicator of public confidence in the police on a yearly basis. Additionally the MPS employ consultations to carry out surveys four times a year into public confidence that is calculated down to LSOA level. This information is not made available to the public. It should be included in a second generation crime mapping system.

Up to now politicians have taken the policy lead regarding crime maps for the public. With police funding cuts looming after the 22nd June 2010 Budget it is time for the police to seize the initiative and demonstrate that police activity does make a difference to communities and individuals. The best way to do this is to introduce radical new second generation crime maps for the public that offer engagement and communication at a local level (7). This paper makes suggestions as to how that may be done.

Footnotes
(1) These are population census areas of an average 633 household and 1500 people. There are 4765 such units in London.
(2) A council ward a census unit which on average contains about 7.5 LSOAs and is also the smallest administrative and electoral unit in England and Wales – each ward has an elected council on the borough council
(3) The MPS police London
(4) “analysis of people by where they live” (Sleight 1997 quoted in Harris, Sleight and Webber 2005 page 2)
(5) the collation of data pertaining to place with reference to people who live and work there.
(6) http://traintimes.org.uk:81/map/tube/
(7) London Community Policing Partnership (LCP2) carries out consultation seminars throughout London twice a year, community engagement and communication between the police and public are consistently rank as the top two topics out of over a hundred. See http://www.lcp2.org.uk/ for more information.

Works Cited
Conservative Party. (2008). Giving The Public A Crime Map Using Technology To Fight Crime. London: Conservative Party.
Guardian Newspaper (2008) http://www.guardian.co.uk/technology/2008/nov/20/ordnance-survey-google-maps
Harris, R., Sleight, P. & Webber, R. (2005), Geodemographics, GIS and Neighbourhood Targeting,
Wiley, London.
Herbert, N., (2010) Minister of State for Policing and Criminal Justice in a speech on the 27th May 2010 at the Ipsis Mori conference entitled The Future of Criminal Justice. The Impact of New Policy, Fewer Resources and Public Influence. Church House Westminster
Hough, M. (2007), Policing, New Public management and Legitimacy in Legitimacy and Criminal Justice, Edited by Tyler, T., New York: Russell Sage Foundation
Hunter A (1978) Symbols of Incivility at the 1978 American Society of Criminology (ASC) conference
Innes, M., (2004) Signal crimes and signal disorders: notes on deviance as communicative action. The British Journal of Sociology 2004 Volume 55 Issue 3
Kelling,G., Pate, A., Ferrara, A., Utne, M., and Brown C., (1981) The Newark Foot Patrol Experiment. Washington: Police Foundation
Noble, M., McLennan, D., Wilkinson, K., Whitworth, A., Exley, S., and Barnes, H., Dibden, C., (2008) English Indices of Deprivation 2007 Communities and Local Government: London
Povey, K. (2001) Open All Hours – A Thematic Inspection Report on the Role of Police Visibility and Accessibility in Public Reassurance, London: Her Majesty’s Inspectorate of Constabulary (HMIC).
Skogan, W (1990). Disorder and Decline: Crime and the Spiral of Decay in American Cities. New York: Free Press
Taylor, R., (1999) The Incivilities Thesis: Theory, Measurement, and Policy, in Measuring What Matters, ed. Robert Langworthy, Washington, DC: U.S. Department of Justice, National Institute of Justice,
UK Statistics Authority (2010). Overcoming Barriers of Trust in Crime Statistics: England and Wales. Monitoring Report 5. London: UKSA
Wilson, J., (1975) Thinking About Crime. New York: Basic Books, 1975
Wilson, J. and Kelling, G. (1982) Broken Windows – The Police and Neighborhood Safety, The Atlantic Monthly, March 1982, 249 29-38.

Future of Policing

2010-07-22T06:51:00.002+01:00

The future of policing is on the agenda again in the context of budget cuts. Sir Denis O'Connor, Her Majesties Chief Inspector of Constabulary (and champion of Neighbourhood Policing) has published "Valuing Policing. Policing in the Age of Austerity " I will be reading it in the next couple of days. The video above is obviously produced in the Netherlands. When I was carrying out Home Office sponsored research in the early 1990s into vehicle crime from the angle of police, insurance, registration authority, manufacturers, MOT etc information systems the Netherlands was about 10-15 years ahead of the UK as far as integration of systems were concerned, so a I expect that this is almost reality now, I will have to visit again soon.

Just a little bit more on violence and recorded crime

2010-07-19T14:28:00.001+01:00

I am going to write another post about police recorded crime and then shut-up for a bit on this subject (unless provoked) and write about the geographic clustering of crime, disorder and policing data - the area that I have been working on recently.

Firstly why is the way police recorded crime is collated and classified important to me? I have a tendency to look at how such things are organised and say that's rubbish and try to reorganise things more logically. The end result may be an improvement but will inevitably have caveats and compromises which need to be explained, and basically it is a big job to do. So I am trying to adopt a more mature attitude and work within the given system. You may notice that I have used Home Office (HO) Crime Type 1 - violent crime against the person when analysing relationships with deprivation. Let me explain the HO Crime Types. If you look at Table 2.04 on pages 31- 35 of this document you will see the police recorded crimes listed. If you really get into the subject you really need to also look at the counting rules here. There are 9 HO Crime Types:

Violence against the Person
Sexual Offences
Robbery
Burglary
Theft and Handling Stolen Goods
Fraud and Forgery
Criminal Damage
Drug Offences
Other Offences

When this list was first compiled, probably over 30 years ago it may have made perfect sense but in that time there have been a miriade of new offences and legislation. The main problem regarding violence is the heiracy of classification gives priority to legislation over whether the crime involved violence. This means that crimes that include violence against the person in their definitions - sexual assaults, robberies, aggravated burglary and aggravated vehicle taking, and others where it is often involved, e.g. kidnapping, rape, arson endangering life are not included in the first category. More bizaarly for the uninitiated is that the most serious public order offences such as riot and violence disorder are not included in the first category but the less serious ones are (under without injury). This is because the more serious offences are not specifically violence against people but violence against everything. In the same way indiscriminate violence such as bombing is hidden in the criminal damage type. Terrorism offences seem to be totally excluded. I could go on but you get the message, a review is long overdue.

Charging a battery with salt ...... that must be wrong

2010-07-18T23:12:00.002+01:00

I used to earn a few extra pennies writing multi-choice exam questions for police promotion exams. This involved devising scenarios where hair-splitting answers could be suggested with only one being the correct. Police Officers would learn legal definitions verbatim from training school onwards so to catch the necessary 25% or more out in an inspectors exam on questions on one of the more well known definitions such assault required splitting of hairs at almost sub-atomic level. All police officers know that the legal definition of an "assault" is "A person is guilty of an assault if they intentionally or recklessly causes another person to apprehend immediate and unlawful personal violence." Battery " is the actual application of that personal violence, e.g. a slap to the face. Thus the Sherlock Holmes type phrase rarely used these days of "assault and battery". It is therefore legally possible to have an injury free assault, that is, an assault without battery.

In the police recorded crime statistics there is a category within the violent crime section of assault without injury. Over the last 10 years about 200,000 such crimes per year have been recorded, about a third of the total assaults recorded. Now if you thought that the offence being recorded was without battery you would be wrong. Appendix A of the Home Office Counting Rules of "violent crime" that can be found here states that these offences include "Slap, Punch, or other attack leaving no visible mark or injury or a passing moment of pain." OK fair enough you may say but in practice these include quite nasty assaults because of the Crown Prosecution Service (CPS) charging rules and the police desire to reduce the more serious violent crime. The charge for this type of assault is "Common Assault" which because it has a maximum imprisonment of 6 months can only be heard in a Magistrates Court. Many of the injuries could fulfill the legal definition of Actual Bodily Harm (ABH) but the CPS will not allow this charge unless they think the likely sentence could be over 6 months imprisonment (very rare) as this can be contested at Crown Court making the process expensive and more likely to end without conviction.

This is just one of many reasons why police recorded violent crime is not what it seems.

Disputing Violent Crime Statistics

2010-07-18T00:01:00.002+01:00

David Cameron blasted for using dodgy crime figures to mislead MPs

Daily Mirror 15/7/10

This is a story that has been rumbling in the background for almost a year now. I have not specifically commented on it up to now because I really need to write a paper of about 20,000 words to explain why everyone is right but at the same time everyone has got certain things wrong. What has prompted me to comment now is that the government says it is going to conduct a review of the crime figures.

Where do I start? Probably best at the publication that is in dispute. This can be found here. This is a Home Office publication explaining the findings of the The British Crime Survey (BCS) and police recorded crime. Ironically, given the fact that it is being slammed by the new government, this is by far the best document of this nature that has ever been produced. For the first time the limitations of the statistics are accurately and comprehensively outlined. This is a grown up document for grown up people. The strength and weakness of the document is that it is researched and compiled by statisticians and quantitative social scientists. They are on firm ground when dealing with the BCS but they do not understand the subtleties of crime definitions and police recording practices of police recorded crime. Leading to the presentation of confusing police recorded crime statistics especially those relating to violence.

This is what the BCS shows regarding violent crime trends, clearly showing a increase up to 1997 when the Conservatives were in power and a decrease since 1997 when Labour was in power. Now this looks all too much like Labour Party spin doctor craftiness to the Conservatives. So they looked deep into the document to find police recorded crime statistics that appear to contradict the BCS figures. I have created a simple graph of these statistics that relate to violent crimes with injury.

The document clearly explains that because of different police counting rules this trend is misleading. The real problem is to do with the haphazard categorisation of crimes that mean that many crimes that are violent are not included in the violent crime figures. Some crimes that most people would not consider violent are. And the whole concept of violence with and without injury is often legally inaccurate. These are areas that the researchers and statisticians appear blind of. There are people in the police who know what is going on but as long as crime figures, especially those relating to violence, are performance indicators for the police it is not in their interest to let on.

I will explain further in future blogs.

London Murder Map

2010-07-16T09:35:00.000+01:00

James Cheshire a fellow PhD student at UCL, his Spatial Analysis blog is here, sent me a link to this murder map of London. It seems to date from the beginning of 2008.

I am always a bit wary of recommending something of this nature if I do not know who is behind it because there are many agendas out there. But having had a look through this site I am impressed with detail of the content and its journalistic style, though I cannot vouch for it accuracy and comprehensiveness it appears authoritive.

It is well worth a look. Click here.

Everything changes but remains the same

2010-07-14T12:41:00.006+01:00

Even though I have been away on holiday - Mauritius, very nice thank you - I have been keeping an eye on developments in policing in the UK following the the General Election.

This is my brief analysis of events. I draw on my experiences of when I was seconded to the Home Office on two occasions so I may have a few useful insights.

First and most importantly we have a new government in power. It is a coalition of Conservatives that tried to maintain in the election that we live in a broken society created by Labour and they will come in to fix it; and the Lib Dems who are ideologically against what they see as the erosion of civil liberties. The right-wing of the Conservatives also are opposed to the big state controlling individuals. So one Labour policy goes immediately - ID cards - a big mistake in the long run my opinion - but it is do with how it was spun by Labour as an anti-terrorist measure - its real benefit is as potentially a net cost saver by allowing state benefits and access to services such as NHS to be properly regulated and targeted for the first time.

Coming back to the allegation of a broken society and supposed rises in violent crime that accompanies the argument. The case was never convincingly made in the Election Campaign and most importantly no alternative strategies were or have been put forward. This may be the reason why the Conservative Shadow Home Secretary Chris Grayling is not the Home Secretary now. The fact is generally Labour had a pretty good record on crime and policing. The move to Reassurance Policing, Neighbourhood Policing and the overarching performance measure of confidence in the police was well researched, logically implemented and funded over a period of about 10 years. This along with the Drug Interventions Programme that recognised the link between heroin, crack cocaine and cocaine addiction and acquisitive crime has made a real difference to many peoples lives.

But (big but) in the Home Office we are talking politics (with a big P, small p and in fact any size of p you can think of). The new government has to be seen to fixing problems of the old regime with new policies. The first thing they have to do has been handed to them on a plate - they have to reduce cost. This was where Labour (and the Home Office) was at their weakest as they thought that the importance of a strategy was reflected by how much money was allocated to it. Actually a lot of what happens in the police only needs a fraction of the money to implement than that which is made available. The money then goes to project managers whose only measure of success is have they spent all the money within the time scales, leading to waste on every level. So there is scope for improvement there which I fully support. This would include a rationalisation of the central police bodies at the top - at the last count there was the National Police Improvement Agency (NPIA), Her Majesties Inspector of Constabularies (HMIC), the Police Standards Unit (PSU) and Association of Chief Police Officers (ACPO), all good sources of employment for retired police officers (so not all bad!); and a rationalisation of oversight in London - the Mayors Office, the Metropolitan Police Authority and Government Office London. This is particularly urgent as the only big new policy that the Coalition Government has come up with so far is the introduction of locally elected police commissioners. This seems to be a Chris Grayling idea which has survived, see here. It is not clear to me how this will all work or even if it is desirable, what I do know is that it will be expensive, add another tier of bureaucracy and take at least 5 years to implement properly by which stage a new idea may be on it way in.

Lastly and probably most importantly for my research is the removal of the overarching performance measure of increasing public confidence in the police announced by the Home Secretary Theresa May, at the ACPO Conference on 29th June 2010. An interesting speech which is worth reading in full. It can be found here. Now this is where in my opinion spin and politics come in. Take this quote from the speech;

But targets don’t fight crime; targets hinder the fight against crime. In scrapping the confidence target and the policing pledge, I couldn’t be any clearer about your mission: it isn’t a thirty-point plan; it is to cut crime. No more, and no less.

The quote makes no logical sense in the cold light of day but thats by the bye. What she (and her advisers) are trying to flag up is judge me by whether crime is cut or not. This is clever politics because this is the one measure that is has been falling year on year and is likely to continue to fall and it is a measure that is totally manipulable by politicians and the police. We are not of course talking about crime here per sae. we are talking about the sub-set which is police recorded crime. In fact there is a very good case to say that rises in police recorded crime is a good thing. For instance, I am sure that police recorded crime in the shanty towns around Recife Brazil (see my previous blog) have a low level of police recorded crime (worth research) but that does not reflect the true situation.

The police service and the Home Office is like a huge oil tanker. It will not be easy or quick to turn round strategies regarding police confidence and they are likely to continue to be measures at local level over-seen by the new local representatives. Local crime statistics and crime maps will become more and more important for police accountability - see also Nick Herbert's speech on 23rd June 2010 here. This will call for a new generation of more sophesticated and comprehensive crime maps in the future. Co-incidently I am presenting a paper at the MapAsia Conference on 27th July 2010 in Kuala Lumpur entitled "Towards a Second Generation of Crime Maps for the Public" which discusses this subject. I will publish it on the blog once it has been delivered.

So what are my are conclusions. Everything changes but remains the same and there is plenty of need for my research out there!

A step too far for police patrols?

2010-06-22T13:44:00.005+01:00

The above is a screen shot of a google map of central London showing in real time the locations of underground trains. You can actually see them move and by clicking on them more information is available. The link is here. I thank John Lahr, a former work colleague for drawing it to my attention.

It occurred to me that a similar website of uniform police vehicle locations and movements is also possible. Police vehicles have mobile data terminals that transmit GPS signals every 15 seconds. I have some of the data that I have plotted for Camden Borough.

This map shows all the locations for a 24 hour period on Saturday 20th March 2010.

The above three maps show the hours of 21, 22, 23 00 hours, some of the busiest times in the week. It may surprise people how little cover there is.

What is not being shown is foot patrols. GPS signals are transmitted from foot patrolling officers personal radios but not stored so I have no access to that data. But it is feasible to plot both vehicle and foot patrols in real time.

Before you cynical people send me to an institution for the criminally naive on the basis that this would give the offenders all the information they need to commit crime undetected let us examine the facts. What is the purpose of police officers patrolling in uniform, often with high visibility jackets, and in livery vehicles with flashing lights and sirens? It is of course to be seen. The purpose of overt policing is to to be identifiable, to prevent crime, to reassure and to engage with the public. There is of course a role too for covert policing.

The more I think about it the more sensible it becomes.............................................................

Fear of Crime & Crime Prevention in Recife Brazil

2010-06-17T10:59:00.006+01:00

Yesterday CASA had an impromptu seminar from Mauro Normando Macedo Barros Filho from Brazil (his blog is here). He gave an interesting talk about the spatial analysis of street robberies in Recife in Brazil.

Recife is an extreme case. It has a frightening crime problem as these links (1), (2), (3) show.

Extreme cases are interesting from a researcher's point of view as it allows the mental testing of hypothesis. In this case my model of fear of crime. I have shared it before but I do so again.

People base their fear of crime on their perception of crime. They apply that perception to their personal circumstances and take action to avoid or prevent that crime happening to them and who they care for. Now in Recife, interestingly even though the fear of crime is very high, wealthy people still want to live there and visit there on their holidays. Crime avoidance becomes lifestyle changing, only going to certain places at certain times in certain ways. And importantly because it becomes observable in the urban morphology is the trend that those who can afford it live their lives behind walls and security systems. The downside is that by making the private spaces safer the public spaces become more dangerous because the natural defences supplied by communal mutual security (people looking out for each other) is eroded. It is the role of the police to make law-abiding citizens feel safe on their streets so they do not have to hide.

I am always on the look out for proxy measures for confidence in the police. Urban defensive design had already occurred to me but perhaps I should add a street based cafe culture which really should exist in Recife but appears not to, to my list.

The interesting case of Home Office Crime Type 1

2010-06-15T23:19:00.002+01:00

I have discussed the index of multiple deprivation in a previous post. The conclusion was that the crime element did not fit with incivility theories because it did not positively correlation with the multiple indices. This why I find this slide exciting. It shows two LSOAs in the borough of Camden that are next to each other. It just so happens that one - 858 is the second most deprived in Camden and 962 is the fourth least deprived. In the scatter graph this is plotted with y axis showing the ranking of LSOA in the Borough of Camden - 1 is the most deprived, 133 the least. The x axis shows the ranking of number of victims (of crime that occurred anywhere in London but mainly in Camden Borough) of Home Office Type 1 crimes (primary violence related crimes) living in each LSOA with the LSOA with the most victims having rank 1 and the least rank 133.

The slide above shows graphs where the victims living in the top twenty ranked LSOAs (group1) for victimisation of Home Office Crime Type 1 crimes are compared with the other 113 LSOA (group 2). It can be seen the that the age, gender and ethnicity distributions are similar. This suggests it is the lifestyle that is aligned to measures of deprivation that are responsible for this variation.

Predictably, if you have read my previous posts, burglary has no similar correlation. If fact the less deprived area of the two LSOAs highlighted has more burglary victims than the more deprived.

Differences in victims

2010-06-14T09:02:00.002+01:00

I use this slide in presentations to show how a similar type of crime can have totally different victim and temporal charateristics; also variations in geographic occurance.

Notes about the future

2010-05-28T08:34:00.000+01:00

I attended an interesting half day conference yesterday organised by Prof. Mike Hough from Kings College London and Ipsos MORI Social Research Institute entitled "The Future of Criminal Justice. The Impact of New Policy, Fewer Resources and Public Influence" I am not a great note taker but these comments were the most significant to me.

Nick Herbert MP, Minister of State for Policing and Criminal Justice.

"We live in an age of accountability" mentioned crime mapping as part of accountability and transparant government. Push towards decentralisation. Long-term fixes (not short term media driven fixes). Holistic approach. No mention of Neighbourhood Policing. Budget cuts inevitable.

Ben Page, Cheif Executive Ipsos MORI

Only 1 person in 100 thinks that we should have fewer police
Anti-social behaviour answers are a good proxy indicator of people's happiness
Only 14% of working class elderly are online

Complexities of Recorded Crime continued

2010-05-25T23:04:00.003+01:00

I have just had a re-read of my previous post and apart from correcting the numerous typos I notice that I did not mention one of the main facets of my argument. One of dangers of writing a post whilst supervising a 20 month old grand-daughter!

This is of course is the fact that we are comparing recorded crime here. One of the differences between those living in high and low deprivation areas is the level of insurance cover. The more affluent, one assumes, will have insurance for household goods and the less affluent will not, a good area for research. Insurance can skew the figures in the same direction by two means. Firstly, as it is necessary to report a crime to police before gaining an insurance pay out, those who are insured a more likely to do so than those who are not. When I carried research into vehicle thefts about 18 years ago I presented evidence that at least 20% of all reported vehicle thefts were in fact insurance frauds. It is only those who are insured who can commit such frauds.

This means that there probably is an under-reporting of burglary in deprived areas and an possibly an over-reporting in affluent ones.

There is still much more I could write on this subject including why I treat police crime data as recording the public's demand and police activity rather being a measure of actual crime.

Complexities of Recorded Crime

2010-05-25T17:43:00.003+01:00

The classfication of recorded crime is complex. I will illustrate this by using the example of "domestic burglary". Burglary is a legal term which in itself is complex and defined in England and Wales by section 9 of the Theft Act 1968. In simple terms it is a theft that occurs whilst trespassing in a premises. If violence is involved then it is an aggravated burglary. Then there are legal definitions around what is and is not an attempted burglary. In legal terms there is no differentiation between domestic and non-domestic burglaries, this is where Home Office crime classifications come in. So a shed burglary in a residential garden is a domestic burglary but a shed burglary in a public garden is a non-domestic burglary. A distraction or deception burglary is classified by the Metropolitan Police Service as burglary artifice. Theft of car keys from a house to steal a car parked outside is a domestic burglary. If the car was stolen without entering the house then it is not a burglary.

My last post I discussed the differences between correlation of domestic burglaries with the IMD in inner and outer London.

The first point to make is that even though the classification of domestic burglary is being compared this includes burglaries of houses, sheds etc., attempted burglaries, aggravated burglaries, burglary artifice, burglaries for cars, high value and low value burglaries. This means that there is no guarantee the nature of the burglaries are exactly the same in inner and outer London. Or even more pertinent the nature of burglaries is the same in higher and lower deprived areas.

Secondly crime commission theories are relevant here too. For a burglary to occur you need the interaction between a likely offender and a premises that they are capable of entering with minimum risk and a commodity to steal that makes their effort worth while. In London with its over 3 million households it only has about 60,000 reported domestic burglaries a year. This means that it only needs about 400 active burglars, commiting an average of 3 burglaries a week for this figure to be acheived.

Thirdly travel to crime theories should also be considered. Most burglars have a modus operandi that works for them. This will involve types of premises they will burgle, time of day and location. The location typically will be within a easy travelling distance from their home or other base but the higher the gain or rarer the desired target the more likely the distance travelled will be greater. Not really a lot different to a hunter/prey scenario.

There is a dynamic here (the Routine Choice Dynamic Theory is something I devised for my M.Phil which I will perhaps outline in a later post) that critically depends on the existance of an active burglar at a location. Therefore information of the locations of active burglars is perhaps equally interesting as where the crimes are occurring.

I could go on mentioning other factors such as drug addiction motivation and the varying performance of police but I will have to save them up for other posts. I will end by concluding that the higher population density and a the greater proximity of more and less deprived areas in inner London compared to outer London probably accounts for at least part of the differences in correlation.