Bullshit about Bullshitters and other Misadventures in Social Science

May 20th, 2019

I recently came across a news story about a social science research study that caught my attention. How could I resist a story about bullshitters? According to the study, titled “Bullshitters. Who Are They and What Do We Know about Their Lives?”, this is “an important new area of social science research.” Reviewing the research paper revealed more about problems in social science research, however, than anything meaningful and useful about bullshit, bullshitting, or bullshitters. In this blog post, I’ll describe a few of these problems.

A Useless Definition

The researchers defined “bullshitters” as “individuals who claim knowledge or expertise in an area where they actually have little experience or skill.” If you read the study, however, you will find that this does not accurately describe the behavior that they examined. A more accurate and specific description would state that bullshitters are “people who claim, for any reason, to be familiar with and perhaps even understand concepts that don’t actually exist.” The study is based on the responses of 15-year old students in English-speaking countries to questions about three bogus mathematical concepts that they answered while taking the Programme for International Student Assessment (PISA) exam. According to this study, students who claim knowledge of a bogus mathematical concept, for whatever reason, are bullshitters. This, however, is not what people typically mean by the term. Typically, we think of bullshitters as people who make shit up, not also as people who make mistakes, but the authors didn’t make this distinction. If you turn to someone and ask, “Are you bullshitting me?,” you are asking if they intentionally fabricated or exaggerated what they just told you. Bullshitting involves intention. The act of intentionally claiming expertise that you lack to inflate your worth in the eyes of others is indeed a behavior that could be studied, but the mixture of intentional deception and unintentional error does not qualify as a single specific behavior.

Why did the researchers define bullshitters as they did? I suspect it is because they couldn’t determine the difference between intentional deceit and confusion about the bogus mathematical concepts. Defining bullshitters as they did, however convenient, produced a useless study. What can we possibly do with the results? Unfortunately, many social science research studies fall into this category. In part, this is a result of the current myopic emphasis in academia on publication. To get ahead as an academic in a research-oriented discipline, you must publish, publish, publish. For individuals, getting published, and for academic institutions, having published studies cited in other publications, is more valuable than useful research. This is a travesty.

Unreliable Measures and Questionable Statistics

By reviewing many social science research studies over the years, I’ve learned that you should take their claims with a grain of salt until you examine the work carefully. To do this, you must not only read the papers closely, you must also examine the data on which the research was based, including the ways the data was manipulated. By “manipulated,” I don’t mean that the researchers intentionally screwed with the data to support particular conclusions, although this does occur, but merely that they produced their own data from the original data on which the research was based, usually by means of statistical operations (e.g., statistical models of various types) that rely on assumptions. To take research conclusions seriously, we must confirm that the data, the statistical models, and the assumptions on which they are based are all valid and reliable. When researchers don’t provide us with the means to validate their data, we should never accept their conclusions on faith. In my opinion, studies that don’t clearly describe the data on which their findings are based and don’t make that data readily available for inspection don’t qualify as legitimate science.

Social science is challenged by the fact that it often cannot directly measure the phenomenon that it seeks to understand. For example, you cannot place a human subject into a machine that’s capable of measuring their bullshitting behavior. You’re forced to use a proxy—that is, to measure something that you believe is closely related and representative—as the best means available. In this particular study, the researchers chose to treat students’ answers to questions about three bogus mathematical concepts as their proxy for bullshitting.

While taking the PISA exam, students were asked about a series of sixteen mathematical concepts, including three bogus concepts—”Proper Number,” “Subjunctive Scaling,” and “Declarative Fraction”—and for each they were asked to select from the following list the response that best described their familiarity with the concept:

    1. Never heard of it
    2. Heard of it once or twice
    3. Heard of it a few times
    4. Heard of it often
    5. Know it well, understand the concept

These five potential responses comprise something called a Likert scale. The items are supposed to represent the full range of possible responses. Another more typical set of Likert items that often appears in questionnaires asks people to assess the merits of something, such as a particular product, by selecting from a list of responses like the following:

    1. Extremely Poor
    2. Poor
    3. Moderate
    4. Good
    5. Extremely Good

A Likert scale is ordinal (i.e., the items have a proper order, in this case from extremely poor to extremely good), not quantitative. Along a quantitative scale, distances between consecutive values are equal. For example, the quantitative scale 10, 20, 30, 40, 50, etc., exhibits equal intervals of 10 units from one value to the next. Distances between items on a Likert scale, however, are not necessarily equal. For example, the difference between “Extremely Poor” and “Poor” is not necessarily the same as the difference between “Poor” and “Moderate.” Also, with the quantitative scale mentioned above, 50 is five times greater than 10, but with the sample Likert scale, “Extremely Good” is not five times better than “Extremely Poor.” In the Likert scale that was used in this study, the distance between “Heard of it often” and “Know it well, understand the concept” seems quite a bit greater than the distance between any other two consecutive items, such as between “Never heard of it” and “Heard of it once or twice.” Likert scales require special handling when they’re used in research.

To quantify people’s responses to Likert scales (i.e., to convert them into quantitative scores), merely taking either of the sample Likert scales above and assigning the value 1 through 5 to the items (i.e., the value of 1 for “Extremely Poor,” etc.) would not produce a particularly useful measure. Researchers use various techniques for assigning values to items on Likert scales, and some are certainly better than others, but they are all pseudo-quantitative to some degree.

Imagine what it would be like to rely on people to determine air temperature using the following Likert scale:

    1. Extremely Cold
    2. Cold
    3. Average Temperature
    4. Hot
    5. Extremely Hot

Obviously, we wouldn’t use a Likert scale if we had an objective means, such as a thermometer, to measure something in a truly quantitatively manner. Subjective measures of objective reality are always suspect. When we convert subjective Likert scales into quantitative scores, as the researchers did in this study, the quantitative values that we assign to items along the scale are rough estimates at best. We must keep this in mind when we evaluate the merits of claims that are based on Likert scales.

Social science research studies are often plagued by many challenges, which is one of the reasons why attempts to replicate them frequently fail. This doesn’t seem to discourage many researchers, however, from making provocative claims.

Provocative Claims

Based on their dysfunctional definition of bullshitters, the researchers made several claims. I found one in particular to be especially provocative: a ranking of English-speaking countries based on their percentages of bullshitters, with Canada on top followed by the USA. As an American, I find it rather difficult to believe that our polite neighbors to the north are more inclined to bullshitting than we are. If we set aside our concerns about the researchers’ definition of bullshit for the moment and accept students’ responses to the three bogus mathematical concepts as a potentially reliable measure of bullshitting, we must then determine a meaningful way to convert those responses into a reliable bullshitter score before we can make any claims, especially provocative claims. Unfortunately, it is difficult to evaluate the method that the researchers used to do this because it’s hidden in a black box and they won’t explain it, except to say that they used an “Item Response Theory (IRT) model to produce a latent construct.” That was the answer that I received when I asked one of the researchers about this via email. Telling me that they used an IRT model didn’t really answer my question, did it? I want to know the exact logical and mathematical steps that they or their software took to produce their bullshitter score. How were the various Likert responses weighted quantitatively and why? Only by knowing this can we evaluate the merits of their results.

Social scientists aren’t supposed to obscure their methods. Given the fact that I couldn’t evaluate the researchers’ methods directly, I examined the data for myself and eventually tried several scoring approaches of my own. Upon examining the data, I soon became suspicious when I noticed that the bogus mathematical concept “Proper Number” elicited quite different responses than the other two. Notice how the patterns in the following graphs differ.

Only the items that I’ve numbered 1 through 4 indicate that the students claimed to be familiar with the bogus concepts. More than 50% of students indicated that they were familiar with the concept “Proper Number,” but only about 25% indicated that they were familiar with each of the other two concepts. Notice that responses indicating increasing degrees of familiarity with “Proper Number” correspond to increasing percentages of students. Far more students indicated that they “Knew it well, understand the concept,” than those who indicated that they “Heard of it once or twice.” This is the opposite of what we would expect if greater degrees of familiarity represented greater degrees of bullshitting. Declining percentages from left to right are what we would expect if students were bullshitting, which is exactly what we see in their responses to the concepts “Subjunctive Scaling” and “Declarative Fraction.” I suspect that this difference in behavior occurred because many students (perhaps most) who claimed to be familiar with the “Proper Number” concept were confusing it with some other concept that actually exists. To test this, I did a quick Google search on “Proper Number” and all of the links that were provided referenced “Perfect Number” instead, a legitimate concept, yet Google didn’t bother to mention that it substituted “Perfect Number” for “Proper Number.” Nothing similar occurred when I Googled the other two bogus concepts. This suggests that people search for “Proper Number” when they’re really looking for “Perfect Number” frequently enough for Google to make this automatic substitution. When I pointed this out to the primary researcher, expressed my concern, and asked her about it in our third email exchange, I never heard back. It is never a good sign when researchers stop responding to you when you ask reasonable questions or express legitimate concerns about their work. If responses to the three bogus concepts were due to the same behavior (i.e., bullshitting), we should see similar responses to all three, but this isn’t the case. In fact, when I compared responses per country, I found that the rank order of so-called bullshitting behavior per country was nearly identical for “Subjunctive Scaling” and “Declarative Fraction,” but quite different for “Proper Number.” Something different was definitely going on.

When I made variously weighted attempts to convert students’ Likert responses into bullshitter scores, I found that, if you consider all three bogus concepts, Canada does indeed take the prize for bullshitting, but if you exclude the question about “Proper Number,” Canada drops below the USA, which seems much more reasonable. As an American living at a time when the executive branch of government is being led by a prolific bullshitter, I can admit, albeit with great embarrassment, that we are plagued by an extraordinary tolerance of bullshitting.

Regardless, I don’t actually believe that we can put our trust even in students’ responses to the bogus concepts “Subjunctive Scaling” and “Declarative Fraction” as a reliable measure of bullshitting. Before I would be willing to publish scientific claims, I would need better measures.

Concluding Thoughts

I was trained in the social sciences and I value them greatly. For this reason, I’m bothered by practices that undermine the credibility of social science. The bullshitters study does not actually produce any reliable or useful knowledge about bullshitting behavior. Ironically, according to their own definition, the researchers are themselves bullshitters, for they are claiming knowledge that doesn’t actually exist. Social science can do better than this. At a time when voices in opposition to science are rising in volume, it’s imperative that it does.

The Data Loom Is Now Available!

May 16th, 2019

After a few months of waiting, my new book The Data Loom: Weaving Understanding by Thinking Critically and Scientifically with Data is now available. By clicking on the image below, you can order it for immediate delivery from Amazon.

Data, in and of itself, is not valuable. It only becomes valuable when we make sense of it. Unfortunately, most of us who are responsible for making sense of data have never been trained in two of the job’s most essentially thinking skillsets: critical thinking and scientific thinking. The Data Loom does something that no other book does—it covers the basic concepts and practices of both critical thinking and scientific thinking and does so in a way that is tailored to the needs of data sensemakers. If you’ve never been trained in these essential thinking skills, you owe it to yourself and your organization to read this book. This simple book will bring clarity and direction to your thinking.

The Smart Enough City: Avoiding the Myopia of Tech Goggles

May 8th, 2019

At this juncture in human history, few issues should concern us more than our relationship to digital technologies. They are shaping our brains, influencing our values, and changing the nature of human discourse—not always in good ways. When we determine our relationship to any new technology, there is a middle ground between the doe-eyed technophile and the intransigent Luddite. Only fools dwell on the extremes of this continuum; wisdom lies somewhere in between. When developing and managing cities—those places where most of us live our lives—wisdom definitely demands something less technophilic and more human than the self-serving visions of “smart cities” that technology vendors are promoting today. Ben Green makes this case compellingly in his thoughtful new book The Smart Enough City: Putting Technology in Its Place to Reclaim Our Urban Future.

In the first few pages of the book, Ben asks the following questions about cities:

Nobody likes traffic, but if eliminating it requires removing people from streets, what kinds of cities are we poised to create?

Nobody wants crime, but if preventing it means perpetuating discriminatory practices, what kinds of cities are we poised to create?

Everybody desires better public services, but if deploying them entails setting up corporate surveillance nodes throughout urban centers, what kinds of cities are we poised to create?

As a whole, the book is Ben’s response.

This book is about why, far too often, applications of technology in cities produce adverse consequences—and what we must do to ensure that technology helps create a more just and equitable urban future.

The term “smart city” has emerged as shorthand for cities that focus on the latest technologies as the solution to human problems. If you buy into this term, you believe that failing to implement the latest technologies is dumb, and who wants to live in a dumb city? It isn’t that simple, however. Technologies indeed offer benefits, but only good technologies, and only when they’re designed well and applied wisely. Framing all of a city’s problems as solvable through technologies ignores the complexities that successful urban development and governance must understand and address. Technology vendors love to promote this reductionist vision of smart cities, but those who actually work in the trenches to make cities livable, just, and equitable recognize a nuanced interplay of forces and concerns that must be considered and coordinated.

Although represented as utopian, the smart city in fact represents a drastic and myopic reconceptualization of cities into technology problems. Reconstructing the foundations of urban life and municipal governance in accordance with this perspective will lead to cities that are superficially smart but under the surface are rife with injustice and inequity. The smart city threatens to be a place where self-driving cars have the run of downtowns and force out pedestrians, where civic engagement is limited to requesting services through an app, where police use algorithms to justify and perpetuate racist practices, and where governments and companies surveil public space to control behavior.

Technology can be a valuable tool to promote social change, but a technology-driven approach to social progress is doomed from the outset to provide limited benefits or beget unintended negative consequences.

Ben calls this problematic perspective “technology goggles,” or simply “tech goggles.”

At their core, tech goggles are grounded in two beliefs: first, that technology provides neutral and optimal solutions to social problems, and second, that technology is the primary mechanism to social change. Obscuring all barriers stemming from social and political dynamics, they cause whoever wears them to perceive every ailment of urban life as a technology problem and to selectively diagnose only issues that technology can solve…The fundamental problem with tech goggles is that neat solutions to complex social issues are rarely, if ever, possible.

Technologies are not neutral and objective; they incorporate values and strive to achieve particular outcomes that can undermine the livable and equitable cities that we desire. Technologies, in and of themselves, are never the solution. Only when good technologies are well designed and used wisely can they contribute to real solutions.

The smart city is thus founded on a false dichotomy and blinds us to the broader possibilities of technology and social change. We become stuck asking a meaningless, tautological question—is a smart city preferable to a dumb city?—instead of debating a more fundamental one: does the smart city represent the urban future that best fosters democracy, justice, and equity?

I believe that the answer is no—that our essential task is to defy the logic of tech goggles and recognize our agency to pursue an alternative vision: the “Smart Enough City.” It is a city free from the influence of tech goggles, a city where technology is embraced as a powerful tool to address the needs of urban residents, in conjunction with other forms of innovation and social change, but is not valued for its own sake or viewed as a panacea. Rather than seeing the city as something to optimize, those who embrace the Smart Enough City place their policy goals at the forefront and, recognizing the complexity of people and institutions, think holistically about how to better meet their needs.

Throughout this book, Ben examines the many ways in which technologies can impact and either assist or harm urban life. He dives deeply into specific issues regarding transportation, police work, civic engagement, and the provision of human services. He examines specific technologies, including autonomous vehicles, sensors, and machine-learning algorithms. He makes his case with example after example, both of smart city failures and smart enough city successes. This story features some bad actors, but quite a few heroes as well. I never imagined that I would find a book about cities so engaging. Even though the book focuses on the ways that technologies are shaping cities—a topic that I haven’t given much thought in the past—the concerns and potential responses that it considers apply much more broadly to technologies and their use. Those of us who are work as technology professionals should heed this book’s wise counsel.

When Ben first approached me and asked if I’d be willing to review his book, I was somewhat apprehensive. As someone who has been writing about information technologies for many years, I am frequently approached by authors with similar requests. More often than not, I don’t end up liking their books well enough to recommend them, and I take no pleasure in telling authors why I made that choice. For this reason, when I encounter a book like The Smart Enough City, I’m relieved. More than relieved, I’m happy to recommend them to my readers. In this particular case, I’m more than happy, I’m thrilled, because this book is an extraordinarily well-researched and well-written treatise on an important topic. The choices that we make about technologies today will fundamentally shape our future. It’s up to us to shape a future that will provide benefit, not oppress.

Turn Up the Signal; Turn Off the Noise

April 21st, 2019

To thoroughly, accurately, and clearly inform, we must identify the intended signal and then boost it while eliminating as much noise as possible. This certainly applies to data visualization, which unfortunately lends itself to a great deal of noise if we’re not careful and skilled. The signal in a stream of content is the intended message, the information we want people to understand. Noise is everything that isn’t signal, with one exception: non-signal content that somehow manages to boost the signal without compromising it in any way is not noise. For example, if we add nonessential elements or attributes to a data visualization to draw the reader’s attention to the message, thus boosting it, without reducing or altering the message in any way, we haven’t introduced noise. No accurate item of data, in and of itself, always qualifies either as a signal or noise. It always depends on the circumstances.

In physics, the signal-to-noise ratio, which is where the concept originated, is an expression of odds: the ratio of the one possible outcome to another. When comparing signal to noise, we want the odds to dramatically favor the signal. Which odds qualify as favorable varies, depending on the situation. When communicating information to someone, a signal-to-noise ratio of 99 to 1 would usually be considered favorable. When hoping to get into a particular college, however, 3-to-1 odds might be considered favorable, but those odds would be dreadful in communication, for it would mean that 25% of the content was noise. Another ratio that is common in data communication, a probability ratio, is related to an odds ratio. Rather than comparing one outcome to other as we do with odds, however, a probability ratio compares a particular outcome to the total of all outcomes. For example, a probability ratio of 85 out of 100 (i.e., the outcome of interest will occur 85% of the time on average), is the mathematical equivalent of 85-to-15 odds. When Edward Tufte introduced the concept of the data-ink ratio back in the 1980s, he proposed a probability ratio rather than an odds ratio. He argued that the percentage of ink in a chart that displays data, when compared to the total ink, should be as close to 100% as possible.

Every choice that we make when creating a data visualization seeks to optimize the signal-to-noise ratio. We could argue that the signal-to-noise ratio is the most essential consideration in data visualization—the fundamental guide for all design decisions while creating a data visualization and the fundamental measure of success once it’s out there in the world.

It’s worth noting that particular content doesn’t qualify as noise simply because it’s inconvenient. Earlier, I said that a signal is the intended message, but let me qualify this further by pointing out that this assumes the message is truthful. In fact, the message itself is noise to the degree that it communicates misinformation, even if that misinformation is intentional. I’ve seen many examples of data visualizations that left out or misrepresented vital information because a clear understanding of the truth wasn’t the designer’s objective. I’ve also witnessed occasions when highly manipulated data replaced the actual data because it told a more convenient story—one that better supported an agenda. For example, a research paper that claims a strong relationship between two variables might refrain from revealing the actual data on which those claims were supposedly based in favor of a statistical model that replaced a great deal of volatility and uncertainty in the relationship, which could be seen in the actual data, with a perfectly smooth and seemingly certain portrayal of that relationship. On occasions when I’ve questioned researchers about this, I’ve been told that the volatility in the actual data was “just noise,” so they removed it. While they might argue that their smooth model illustrates the relationship in a simpler manner, I would argue that it over-simplifies the relationship if they only report the model without also revealing the actual data on which it was based. Seeing the actual data as well helps us keep in mind that statistical models are estimates, built on assumptions, which are never entirely true.

So, to recap, noise in communication, including data visualization, is content that isn’t part of and doesn’t support the intended message or content that isn’t truthful. Turn up the signal; turn off the noise.

Worthy of Your Attention

April 9th, 2019

I spend a great deal of time reading books. Many of them cover topics that are relevant to my work in data sensemaking and data visualization, and most of them are quite good, but only a few are extraordinary. The new book, How Attention Works: Finding Your Way in a World Full of Distraction, by Stefan van der Stigchel, definitely qualifies as extraordinary.

Stigchel is a professor in the Department of Experimental Psychology at Utrecht University in the Netherlands. Until recently, I taught annual data visualization workshops in Utrecht for several years. Had I known about Stigchel at the time, I would have definitely invited him out for a beer during one of my visits. His work is fascinating. This book focuses on a specific aspect of visual perception: visual attention—what it is, how it works, how it is limited, and how it has allowed the human species to progress beyond other species. It does so in a practical manner by explaining how an understanding of visual attention can improve all forms of information design.

I only know of one other author who has written practical works about visual perception with such clarity and insight: Colin Ware, Director of the Data Visualization Research Lab at the University of New Hampshire. It was from Ware’s two books—Visual Thinking for Design and Information Visualization: Perception for Design—that I learned much that I know about visual perception and its application to data visualization. Although Stigchel doesn’t address data visualization in particular, what he reveals about visual attention complements and, in some respects, extends what Ware covers in his books. Here’s an excerpt from the preface that will give you an idea of the book’s contents and intentions:

If you dig deeper into the subject of visual perception, you will quickly discover that we actually register very little of the visual world around us. We think that we see a detailed and stable world, but this is just an illusion created by the way in which our brains process visual information. This has important consequences for how we present information to others—especially attention architects.

Everyone whose job involves guiding people’s attention, like website designers, teachers, traffic engineers, and, of course, advertising agents, could be given the title of “attention architect.” Such individuals know that simply presenting a visual message is never enough. Attention architects need to be able to guide our attention to get the message across…Whoever can influence our attention has the power to allow information to reach us or, conversely, to ensure that we do not receive that information at all.

Everyone who visualizes data and presents the results to others is an attention architect…or should be. To visualize data effectively, you must learn how to draw people’s attention to those parts of the display that matter and to prevent the inclusion of anything that potentially distracts attention from the message. You can only do this to the degree that you understand how our brains manage visual attention, both consciously and unconsciously. Reading this book is a good start.

“The Data Loom” can now be ordered!

March 14th, 2019

Even though my new book, The Data Loom, will not start shipping until early to mid May, it can now be ordered from Amazon. If you haven’t already read about the book, please take a look at my previous blog post.

Digital Thoreau

February 13th, 2019

In 1854, Henry David Thoreau’s thoughtful account of his years in the woods at Walden Pond was published. The book, Walden, is filled with insights that could only be acquired through quiet reflection on life’s essentials. One of my favorite quotes from the book, which I’ve used often in my work, is “Simplicity, simplicity, simplicity.” Even back then in the mid-19th century, long before the distractions of our modern digital world, Thoreau recognized the importance of choosing how we spend our time with great care. The value of his cautionary guidance is even greater today, for the digital distractions that vie for our attention are more prolific, insidious, and potentially harmful than those that Thoreau encountered. In the spirit of Thoreau, Cal Newport has written a wonderful new book to help us live with greater intention and less distraction in the modern world, titled Digital Minimalism: Choosing a Focused Life in a Noisy World.

Newport has rapidly become one of my favorite thinkers and writers about technology. Back in 2016, I reviewed one of his previous books, Deep Work, which I dearly love. Newport and I are both digital professionals who approach technologies with careful consideration. Neither of us are anti-technology. Instead, we understand that technologies are not inherently good, and that we should only embrace those that are truly useful and only do so in a way that preserves that usefulness without inviting waste or harm.

Several thoughtful writers in recent years have pointed out how digital technologies have been designed to harvest our attention for profit. Tim Wu, who wrote The Attention Merchants, is perhaps foremost among them. Newport addresses this concern by providing a prescription for managing the harmful effects of digital technologies. He calls this prescription digital minimalism, which he defines as:

A philosophy of technology use in which you focus your online time on a small number of carefully selected and optimized activities that strongly support things you value, and then happily miss out on everything else.

His prescription goes beyond tips and tricks, such as occasional digital sabbaths. It is more thorough, as the situation demands.

The problem is that small changes are not enough to solve our big issues with new technologies. The underlying behaviors we hope to fix are ingrained in our culture, and…they’re backed by powerful psychological forces that empower our base instincts. To reestablish control, we need to move beyond tweaks and instead rebuild our relationship with technology from scratch, using our deeply held values as a foundation.

Digital technologies are shaping cultures and minds, often in harmful ways. Before the words “Simplicity, simplicity, simplicity” in Walden appears the sentence “Our life is frittered away by detail.” At no time in the past has this been truer than it is today. The constant ding of incoming text messages, barely informative “Likes” on Facebook, and the endless queue of tweets that tether us to our smartphones are frittering our lives away. Digital technologies can add great value to our lives, but not if we embrace them indiscriminately.

Announcing “The Data Loom”

January 14th, 2019

When I wrote my most recent book, Big Data, Big Dupe, in early 2018, I thought it might be my last. As it turns out, I was mistaken. Mid-way through 2018, I became concerned enough about a particular problem to write another book, titled The Data Loom: Weaving Understanding by Thinking Critically and Scientifically with Data.

To give you an idea of its contents, here’s the text that will appear on the book’s back cover:

Contrary to popular myth, we do not yet live in the “Information Age.” At best, we live the “Data Age,” obsessed with the production, collection, storage, dissemination, and monetization of digital data. But data, in and of itself, isn’t valuable. Data only becomes valuable when we make sense of it.

We rely on “information professionals” to help us understand data, but most fail in their efforts. Why? Not because they lack intelligence or tools, but mostly because they lack the necessary skills. Most information professionals have been trained primarily in the use of data analysis tools (Tableau, PowerBI, Qlik, SAS, Excel, R, etc.), but even the best tools are only useful in the hands of skilled individuals. Anyone can pick up a hammer and pound a nail, but only skilled carpenters can use a hammer to build a reliable structure. Making sense of data is skilled work, and developing those skills requires study and practice.

Weaving data into understanding involves several distinct but complementary thinking skills. Foremost among them are critical thinking and scientific thinking. Until information professionals develop these capabilities, we will remain in the dark ages of data.

This book is for information professionals, especially those who have been thrust into this important work without having a chance to develop these foundational skills. If you’re an information professional and have never been trained to think critically and scientifically with data, this book will get you started. Once on this path, you’ll be able to help usher in an Information Age worthy of the name.

And here’s an outline of the book’s contents:

Chapter 1 – Construct a Loom

Data sensemaking—the ability to weave data into understanding—requires a spectrum of skills. Critical thinking and scientific thinking are foremost among them.

Chapter 2 – Think Critically

When we think critically, we apply logic and avoid cognitive biases.

Chapter 3 – Think Scientifically

When we think scientifically, we apply the scientific method.

Chapter 4 – Question the Data

Thinking critically and scientifically leads us to ask essential questions about data to improve the reliability of our findings.

Chapter 5 – Measure Wisely

Metrics can be powerful, but we often measure the wrong things, measure the right things ineffectively, and use measurements in harmful ways.

Chapter 6 – Develop Good Thinking Habits

In addition to critical thinking and scientific thinking, data sensemaking is also enriched by developing good thinking habits.

Chapter 7 – Develop a Data-Sensemaking Culture

Effective data sensemaking is undermined by most organizational cultures. We must promote the cultural changes that are needed to embrace critical and scientific thinking with data.

Epilogue – Embrace the Opportunity

The Data Loom is scheduled for publication in May by Analytics Press.

The Malady of Lost Connections

August 14th, 2018

I just finished reading the most important book that I’ve encountered in years: “Lost Connections,” by Johann Hari. It succeeds in doing what its subtitle claims: “Uncovering the real causes of depression—and the unexpected solutions.”

As one of the many people who have struggled with depression, I greatly appreciate the insights that Hari shares in this book. My appreciation extends well beyond this, however, for this book isn’t just about depression and anxiety. It is also a thoughtful and thoroughly researched assessment of modern society. Depression and anxiety are symptoms of deep and systemic flaws in our modern, industrialized, consumerized, and technologized world, which has caused us to lose vital connections that are essential to human fulfillment. As it turns out, depression and anxiety are clear signals that something is very much amiss with our world.

If you collect all of the research data regarding anti-depressants—not just what the pharmaceutical industry has made public—and assess it without bias, you will find that depression is not the result of a chemical imbalance. There is no credible evidence that boosting serotonin actually reduces depression beyond the placebo effect. Furthermore, even though research indicates that some genes can predispose us to respond to our circumstances with depression, the causes of depression do not reside in human biology. Rather, they are rooted in human society and, in some cases, in traumatic experiences. Nevertheless, depression and anxiety are almost always treated by the medical community with drugs that are designed to correct a chemical imbalance that isn’t the cause. This approach has failed miserably.

As the book’s title suggests, depression and anxiety are rooted in disconnections:

  • Disconnection from meaningful work
  • Disconnection from other people
  • Disconnection from meaningful values
  • Disconnection from childhood trauma
  • Disconnection from status and respect
  • Disconnection from the natural world
  • Disconnection from a hopeful or secure future

It took Hari several years to track down the data and interview the experts, resulting in an incredible story. These disconnections are intricately woven into the fabric of modern society. Nevertheless, there are still places where depression and anxiety are rare. In those places still exist the connections that have been disrupted elsewhere.

There are steps that we can and should take as individuals to reestablish the connections that are vital to our lives, but the full solution lies in societal change. Hari lays out many of the steps that we can take to make this happen. Societal change isn’t easy and it takes time, but it’s the only thorough and lasting solution. The change that’s needed doesn’t require the rejection of useful advances in science and technology, but we must embrace these artifacts of modernity more intelligently and with greater care.

Please read this book. Please contribute to the restoration of connections in society that humankind sorely needs to endure and thrive.

The Perils of Technochauvinism

August 1st, 2018

More and more these days people are waking up to the fact that digital technologies often fail us and sometimes do great harm. The default assumption that digital technologies are always needed and beneficial is now being questioned by an increasing number of thoughtful people who understand these technologies well. One such person is Meredith Broussard, who, in her new book, Artificial Unintelligence, labels this erroneous assumption technochauvinism.

“Technochauvinism” is roughly equivalent to Evgeny Morozov’s term “technological solutionism,” which I’ve been using for years. Better than other writers so far, Broussard explains the nature of digital technologies—what they are, how they work, what they do well, the ways in which they’re limited, and how they fail—in a manner that’s practical and accessible to anyone who’s interested. As an accomplished journalist, her writing is clear and rooted in evidence. As an experienced digital technologist, what she says is well informed.

As the title suggests, much of this book focuses on artificial intelligence (AI), which is fitting given the prolific hype and common misunderstandings that obscure AI technologies in particular. Broussard explains what artificial intelligence is and isn’t. While others have described the important distinction between general AI and narrow AI, Broussard explains this difference more clearly and illustrates AI more realistically, using interesting examples. In one chapter, she walks readers through the use of algorithms to make sense of who survived the Titanic disaster, and in so doing reveals both the strengths and weaknesses of machine learning. In another chapter, she takes readers along on a ride in an autonomous vehicle to illustrate the dangers of AI that overreaches. She puts the proper application of AI into perspective.

Apart from AI in particular, she also describes the historical roots of technochauvinism as a byproduct of the worldview that is shared by most of high tech’s power elite. When this limited, self-serving worldview is incorporated into digital technologies, problems result, often promoting injustice.

Computers compute—they do math. As such, they’re better than humans at doing tasks that are based on mathematical computations. Despite the ubiquitous metaphor, computers are not like human brains. Computers don’t think, they aren’t sentient, even though terms such as artificial intelligence and machine learning suggest otherwise. Computers excel at computationally-based tasks, but we can’t rely on them to perform tasks that require understanding, which they lack, without humans in the loop.

This book will not be well received by those who are so invested in digital technologies that they refuse to think critically about them. I’ve already noticed a few undeserved, negative reviews of this book on Amazon that reflect this closed-minded, self-serving perspective. Writing this book took courage. You don’t write a book like this to gain popularity or make money. You do it because you care deeply about the world. This book speaks the truth—a truth that needs to be heard.