Stop Writing Novellas – The People Want Novels

I stumbled upon the most amazing study. Actually, there are a lot of studies on this site, but this most recent one is so huge and comprehensive, you’ll probably be too exhausted after reading it to bother looking at the rest: May 2016 Author Earnings Report: the definitive million-title study of US author earnings

It’s daunting, so I’ll give you a quick sketch of what they did: They took a one-day snapshot of everything happening in the book business on Amazon.com. Everything. And from that, they extrapolated to make a lot of really interesting conclusions. These are serious data scientists, and they have plenty of evidence that extrapolating from a single day is a legit approach.

Here are some of the things I found most interesting:

  • There are only about 1000 authors making a professional salary writing books.
    The odds of making into the NFL are almost twice as good as your odds of being able to make a professional salary as a book author.
  • Almost all book authors who make a professional salary are independent.
    Your chances of making a professional salary as an independent are really bad, but your chances writing for the traditional or small presses are basically nil.
  • There are only about 2500 authors making enough writing books that their spouse might not insist they get a second job.
  • Big 5 publishers make 40% of the revenue, vs 24% going to indies; but
    Big 5 authors make 22% of the royalties, vs 47% going to indies.
    You already knew that the traditional publishing industry screws the authors, but now you have proof.

I asked the authors of the study a few questions in the comments section, and they got right back to me. Check out this chart they produced:

Source: autherearnings.com

Source: authorearnings.com (reproduced here by permission)

You may recall that I mentioned Kindle “15 Minute Reads” in the piece about my short story. There are a lot of really short books with inflated prices, and I was wondering if they might skew the numbers. He gave me this chart to prove that it doesn’t matter, because nobody is buying those or borrowing them.

For those of you who aren’t so good with charts, let me explain this one to you. It says that 60% of the books people buy (including reading on KU) are novels (200+ pages). But 75% of the books people write are novellas and short stories (less than 200 pages). People have no interest in your lazy-ass novellas, people. They want novels.

Makes me glad I happen to write novels. Another fortunate turn of events for me is that I write romance novels. It turns out that romance accounts for about a third of all royalties earned. Other genres account for about 10% each.

There is a lot more stuff in that study. I encourage you to read it, including the comments section. Fascinating stuff.

Oops – I Did it Again

I wrote another web app. I’m friends with a bunch of “branded” Twitter comedians. And they were complaining about people “meme-ing” their tweets. That is, making a picture with their words on it. More often than not, these nefarious people omit attribution (so it’s just plagiarism), put their own branding on it (fuckers), and put it on Facebook where it gets tens of thousands of thumbs-up. That sucks, and it makes my friends mad.

So we were talking about it, and I noted that Twitter isn’t helping matters. If you share a link to a tweet on Facebook, it looks awful. You’ll get a super-blown-up picture of the person’s AVI and only part of the tweet. Nobody is going to put that hot mess on their Facebook. And while it’s unlikely we can get Twitter or Facebook to improve that situation, it seemed to me that it would be pretty easy for an intermediary to improve things quite a bit.

So I wrote an app. twitter-me.me

It’s like Twitter Meme, get it? You put a tweet link into it, and it turns it into a nice picture. Originally, just white words on a dark blue background. Although I’ve since updated it to give you a choice of colors and fonts and layouts. And it gives you a new link, which is really just the link you started with, except you put “http://twitter-me.me” in place of “https://twitter.com”. (Supporting https is absurdly expensive for a free app, so I’m not doing that until there’s a robust way to do https for free.)

Tweet shared as a picture on Facebook

Tweet shared as a picture on Facebook

When you share a link on Facebook, it goes and looks at the web page to find the title and description and picture. I set up the app so that when Facebook looks at that link, it gets the @ of the person who tweeted it as a title, the tweet itself as the description, and the picture of your words as the picture. You don’t need to do anything special. Just share that link and it looks great.

If somebody uses a browser to view the link, though, it serves up a “redirect” that just takes you right to the tweet. So the result is that if someone clicks on that picture, they end up on the tweet and and can like it or retweet it or follow you or whatever.

Tweet shared as a picture on tumblr

Tweet shared as a picture on tumblr

That part was actually pretty easy. So then someone asked me if I could make the same thing happen with tumblr. Remember tumblr? It’s where the porn was until Yahoo! bought it. Anyway, some people still use it, and one of them was wondering if they could share these tweet-pictures there. Tumblr has a “share” button that makes it easy to share a picture, but there is no option on it to include a link. You can have a link attached to a picture. But you’d have to go manually set it up, which is a drag.

So I dug deeper and found that tumblr has an API, so other web applications can post to tumblr on behalf of a user. So after spending some time fighting with that, I managed to implement a tumblr share button that works just like Facebook. You share the tweet to your blog, and it looks nice and if the viewer clicks it, they end up at the tweet.

Tweet shared as a picture on Twitter

Tweet shared as a picture on Twitter

So the next thing that happened was someone asked if they could share the picture on Twitter. Why would you want to share a picture of a tweet on Twitter, instead of just RT’ing the tweet itself? Beats me. But it was super easy, so I did it anyway. Twitter works the same way Facebook does when you share a link. It goes to that URL and sees what kind of a preview it can generate. I just had to recognize that “twitterbot” was looking at the link, and in that case serve up some special tags that let it know to show a nice picture.

So now you can take your tweet, make it look nice as a picture, and share it on Facebook, tumblr, and Twitter. I don’t think there’s any way to get Instagram in on this, because they don’t let you attach a link to a picture. And making pictures that link back to the tweet is kind of the whole point. But I’m open to suggestions.

I also made a page where you can see the most popular (based on click-through counts) of these image: twitter-me.me/browse Each one has a button to let you share it to your Facebook or tumblr accounts. If this takes off and lots of people start making these, I’ll probably add more features to that, like viewing “trending” or “newest” or whatever. Right now, I just show the top 20, like a leader board.

I’m hosting this in Google App Engine, which has pretty decent free quotas. So it isn’t costing me anything (except a few bucks for the domain name). If it really takes off, and I can’t run it for free any more, I’ll figure out some painless way to monetize it. But in my experience, things usually don’t take off. So it’ll probably just remain a free service my friends can use to put their tweets on Facebook. And I’m perfectly happy with that.

Ironically, I can’t use it myself, because I don’t want Facebook people to know about my Twitter account.

Stop Paying People to Read your Book

In marketing to consumers, there is a well-established “buying cycle.” There are a lot of different variations on this but they generally go:

  • Awareness (finding out your product exists)
  • Research (figuring out whether they want it)
  • Purchase (woo hoo!)
  • Repurchase (they liked it and want another)

I mention this because the business of marketing a book is really no different from the business of marketing anything else to consumers. What I find interesting is that the people marketing books these days are mostly authors, and judging from their behavior, I think many of them are really confused about that whole cycle. So I’m writing this post to help explain it to them, with they hope that they stop throwing their money away solving problems they do not have.

Let’s skip awareness for a second, and dispense with the rest of the cycle.

If you write a great book and get a few people to review it, then you’ve got the “Research” step locked. People will look at the reviews and decide to buy it or not. It’s a meritocracy, and we all love those, right?

“Purchase” is not a problem in the book business. Amazon makes that easy.

“Repurchase” is about getting people to want the sequel. Write a great book, and you get that, too.

The really difficult part in the book business is “Awareness.” The number of new books introduced every year is staggering (one source I found says it’s more than a million). Obviously, you need to get the word out about your book, so people will want to learn more. You need them to be interested enough in your book that they will read the reviews. So how do you build awareness? Advertising. Period. There is no other way to build awareness. And this is the point I think authors are confused about.

I’m advertising my book by tweeting to my followers. I am sure to mention my book whenever I am DM chatting with a new follower. (I never send DMs just about my book. I mean that if I’m already chatting with someone, I make sure I mention it at some point in the conversation.) Every single sale I’ve made has been due to either direct advertising by me, somebody advertising on my behalf by retweeting something I put on my account or one of my character accounts, or somebody I advertised to recommending it to a friend (or outright buying it for that friend). Every. Single. Sale.

I am quite certain that not a single person has bought my book because they “discovered” it on Amazon’s web site.

I’ve been watching the progress of Ana Spoke with great interest. She is marketing her own book that she finished about the same time I finished mine. And she posts all the results of the promotions she is doing. The way book promotion usually works is: You run a $0.99 sale (on the Kindle version) and place ads in newsletters to get people to go buy your book while it’s on sale. Some of the newsletters run these ads for free, and others charge short money—typically $25 to $60 for a placement. If you spend $50 on an ad, and Amazon takes 30% of your $0.99 sales price, you need to sell 72 books to break even. News flash: 72 is a hell of a lot of books. (The average book sells 250 copies a year, and that number is skewed up by a few blockbusters.)

Ana just finished a string of three promotions to measure the effectiveness of this strategy. She didn’t break even in any of them. She did sell a lot of books. She shot up into the top 10 in one of her subcategories for a few hours, and she stayed in the top 20 for a couple days. So even if she didn’t break even, it was money well spent, right?

Wrong.

Let’s go back to the buying cycle. We established that the hard problem is “Awareness”—how do you get people to know about your book. If you can advertise for free (by tweeting or in newsletters that don’t charge to advertise a $0.99 sale), that directly addresses awareness. They see the ad, they like the cover, they go look at reviews. But if you pay for the ad, and you don’t get enough buyers to cover the cost, you are losing money. That might be okay if you already have a sequel, and you are pretty sure they will buy that the moment they finish book one. But if you don’t have a sequel yet, that’s just money you are throwing away. You are paying people to read your book. Stop that!

The delusional thinking that I see over and over from authors is that having a good “bestseller” ranking in their niche is going to increase the visibility of their book. And so people browsing for books will see it, and buy it. There are two problems with this thinking:

  1. Nobody browses for books by best seller ranking. Particularly ranking in some random sub-category three levels deep.
  2. Even if they did, they certainly aren’t going to look past the top 10.

Go look at the Kindle e-books web site. I’ll wait. Did you even see the link to browse by best seller ranking? It was there, but it was pretty well hidden. And even then, it showed you the top 100 books overall. Not the top 100 in some random subcategory. There is no way somebody is going to go digging through subcategories to find out what’s selling well. Why would someone do that? If you scroll down the page, you’ll see the top 10 selling books overall. No amount of promotion is going to get you into that list.

So here’s the bucket of cold water: Your Amazon Ranking Does Not Matter.

It would matter if you got into the top 10 overall, but that will never happen for your book.

Sorry.

So the other thing I keep seeing is authors begging for more reviews. There is a widely held belief that something magical happens when you get 50 reviews for your book. I searched and the only source I could find for this myth is this blog piece. In that, the author says that Amazon considers featuring your book if you have more than 50 reviews. However, she provides no evidence to support the contention, and it is clearly not true. Go to that Kindle e-books link I gave above, but use a “Private Browsing Window” so Amazon has no idea of your purchase history. Now you are seeing the stuff they think the general public will buy. Look at the number of reviews as you scan down the page. I’ll wait.

Here’s what I see: 32 84 0 19 184 24 63 39

And then lots with thousands of reviews. Clearly, 50 is not a magic number.

Don’t get me wrong—getting more reviews of your book is great! It definitely helps with the “Research” step of the buying cycle. But it does not do anything for “Awareness,” which you recall is the only problem you have.

Getting people to write reviews is relatively easy, so that’s what we authors ask them to do. And then we conjure a mythology that if we just get enough reviews, we will solve the awareness problem. But there is no evidence at all that this is true. Just as there is no evidence that having a bestseller ranking in some random subcategories will raise awareness. Of course it won’t. Have you personally ever browsed books based on which one are selling well? No, you haven’t. If you browse at all, you are looking at editor picks, and those are there because of name recognition of the author or inside dealing (Amazon pimps the books they themselves publish) or graft and corruption. You aren’t going to be an editor pick no matter how many reviews you get. It isn’t going to happen.

Another myth I’ve seen repeated frequently is that having a lot of reviews or a good bestseller ranking makes your book appear when the user searches, or in the “similar to” list. This is also demonstrably not true. For example, as I’m writing this, Beth Teliho is running a $0.99 promotion for her absolutely wonderful book Order of Seven. (Go buy it; I’ll wait.) Her promotion is going great. She’s selling a lot of books. Right now I see that she is #2 and #3 in some subcategories. 6,659 overall rank. She has 82 reviews, averaging 4.9 stars. You wish your book was in this position!

So let’s go to that private browsing window and search for “Young Adult Adventure” and see if she comes up. Nope. What does come up? Number of reviews (in order that Amazon listed the books): 54 87 14 37 115 0 0 59. Number of stars (same order): 0 4.2 4.3 0 4.4 4.0 4.1 0 0 4.2. Bestseller rank (overall, same order): 777; 544; 36; 147,573; 1,058; 2,648; 3,571; 122,471; 3,128; 52,020

The only conclusion you can draw is that however the fuck Amazon is picking these books, it’s not based on reviews or ranking.

Just to emphasize how much your bestseller rank does not matter, I refined my search. Her book is #2 in eBooks > Teen & Young Adult > Historical Fiction > Ancient Civilizations and #3 in Books > Teens > Historical Fiction > Ancient. I searched “Young Adult Ancient” and “Young Adult Ancient Civilizations” and her book did not appear in the first 100 listed. I searched the exact name of the category “Teens Historical Fiction Ancient” and her book came up #79. Think about that: The #3 book in the category named exactly what I just searched is the 79th one Amazon thought they should list!

So let’s review:

  1. Stop thinking having a lot of reviews helps get you discovered. That’s simply not true.
  2. The only benefit to running an ad campaign is the direct sales you get from that campaign. The boost in your bestseller rank does not help you get discovered.

If you are making $0.70 for every book you sell at $0.99, then you had better be spending less than $0.70 for each buyer. You must get 14 sales for every $10 of advertising spent just to break even. If you get fewer than that, you are paying people to read your book.

For the vast majority of authors, the only cost effective strategy for awareness is free advertising. Twitter, Facebook, and newsletters that promote $0.99 sales for free. The number of reviews doesn’t matter. Bestseller rank doesn’t matter. Anything that costs you money is paying people to buy your book.

If you are an author, and you are paying people to buy your book, cut it out. All you are doing is propping up an advertising machine that is overcharging everyone. Demand that you get more than 14 buyers for every $10 you spend. If we stop paying these ridiculous rates for ineffective ads, the rates will come down. And stop justifying your overspending using myths that are demonstrably untrue. Stop it. Just. Stop it.

My Science of Parenthood Book Report

Science of ParenthoodA while back, a few of my tweets were chosen to be in a book called The Bigger Book of Parenting Tweets. That compilation was put together by Kate Hall, Norine Dworkin-McDaniel, and Jessica Ziegler, which is how I came to know these incredibly cool people. Norine and Jessica have just published another book, Science of Parenthood: Thoroughly Unscientific Explanations for Utterly Baffling Parenting Situations which is full of cartoons and funny stories and charts and graphs and stuff. They use science as a jumping off point to make funny observations about parenthood.

It’s a really good book. Click the picture and it’ll take you to Amazon and you can buy it. I’ll wait.

doo be doo be doo be doo…

Okay, you’ve ordered it? Cool. So what immediately struck me about this book was that there was no way that anyone would get the set-up of these jokes. There are all these oblique references to laws and principles and such that ordinary people wouldn’t know. Show of hands, who knows what Hooke’s Law is about? (Sees one hand; no, it has nothing to do with Peter Pan; sees no hands.) Exactly.

Yet I’m reading all these reviews and everybody thinks the book is really funny, so I guess maybe you don’t need to completely understand the joke setups for them to work. But I figured I’d go ahead and dig deeper into one of these, so you can see what you’re missing.

Mach’s Date-Night Principle
A baby will exert a gravitational pull on a new Mom so intense that despite numerous text messages assuring her that the baby is, in fact, still breathing, she will need to race home to check for herself.

Alright, so you think that’s funny, because it’s true, right? But why did they call it “Mach’s Principle”? Anyone? Anyone?

BRUCE LIPSKY/The Times-Union--11/12/10--[PHOTO GALLERY]--Passengers scream on the Zero Gravity ride. The fair weather made for a fun day at the Greater Jacksonville Agricultural Fair Friday, November 12, 2010, in Jacksonville, Florida. (Florida Times-Union, Bruce Lipsky) 2010

Wheel! Of! Vomit!

Mach was trying to explain centrifugal force. You get in that big wheel-shaped thing at the fair, and it spins around and you are pinned to the wall, and then it tilts up and you stay pinned to the wall instead of falling toward Earth. Why is that? You might not have noticed, but Earth is really huge. Its hugeness gives it a giant gravitational pull. It’s strong enough to keep the moon in place. But yet that silly spinning wheel thing at the fair is able to defeat it. Why?

The really funny thing is that there isn’t a great explanation. Newton just threw up his hands and said “law of nature.” Mach thought that was kind of lame, so he came up with this explanation that it had to do with frames of reference. I’ll try to explain his thinking.

When you are in that wheel thing, you can look across at the other people and they don’t seem to be spinning, right? Within your frame of reference, they aren’t spinning. Everyone is just standing in exactly the same place. Kind of like how we don’t think of other things on Earth as spinning, even though we know from school that the whole damn Earth is spinning. Within our frame of reference, we’re just sitting still.

Okay, so that’s cool, but then why can’t you walk across the wheel to the other side? If it’s spinning fast enough, you probably can’t even lift your arms from against the wall. So within your frame of reference, there’s this massive “force” pulling on you. Suppose that wheel was floating free in space and there was nothing else in the whole universe. Would you still feel that force?

Think about it. The motor that’s driving the wheel has nothing to “hold on to” right? So it’s spinning relative to you, and you’re spinning relative to it. But you’d be able to walk across that wheel in that case, right? Everyone could step away from the wall and walk to the middle and look down at that big motor which is just spinning away, and then look at each other and realize they have no air to breathe and die.

So Mach thought about that, and figured the reason for that force was because you were moving relative to other stuff. Specifically, “distant stars.” He figured there was a big universal frame of reference, and you felt the force because you were moving relative to that. This is the same guy who said the idea of “atoms” was ridiculous, so don’t put all your money on Mach being right about this. But that was his idea, and it gave Einstein the idea that maybe everything is relative, even time. But that’s a post I already wrote. (That’s a link to a post about my scrambled egg recipe in which I inexplicably manage to also talk about general relativity.)

Mach’s Principle therefore, is that stuff “out there” defines the forces we feel “in here.” And so if you go look at the joke (from the book—remember, this is my book report), it’s clever because the force “out there” is not the distant stars, but rather the baby. And the “in here” is not our inertial frame of reference, but rather our personal sense of where we need to be. So there’s a really clever joke-inside-the-joke there that you didn’t even get! But now you do.

So what I’m saying is, this is a really funny book. But it’s even funnier if you take the time to go look up the laws and principles they are referencing, because there’s a whole ‘nother level of brilliant waiting there for you if you do!

I know you already bought the book because I told you to up top, but if you now want another copy to give to your friends, here’s the link again.

Crowdsource Editing

When last we met I was about to send my novel Entropy (which apparently, I should have been calling a “manuscript”) to a genuine book industry book editor. The hope was that she would give me an idea of what kind of a book I had written, and what the right next step was. I think what I learned was more about the publishing industry, than about my book, per se. But I learned a lot!

I found out that my prose was good, and my grammar was good, and all my novel needed was a complete rewrite and thousands of dollars of professional editing. Wait, what? Yes, she explained to me that my manuscript was exactly like every other “first draft.” They all need a complete rewrite. That’s just the way it is. But wait, it gets better.

So after I spend five thousand dollars or so having someone else completely rewrite my book, I might get an agent to look at it. If I’m lucky. And if I get one to take it, I might get a publisher to look at it. If I’m lucky. And if I get a publisher, they are going to—wait for it—hire another editor to completely rewrite the book again.

So I have a book that I thought was pretty close to ready for publication, and the traditional print industry wants me to spend a year and thousands of dollars to transform it into a completely different book that they will be able to sell lots of. Well not sell, exactly. Publishers are apparently no longer in the business of actually selling books. No, I’m told the authors need to do that themselves. If you don’t actively market your own book, they won’t either, and the book won’t sell.

Also, if you’re lucky they’ll give you 15% of the sales, except they don’t really because they charge you back for lots of things. And the agent gets a cut. Plus you’re out the $5K you spent on that edit that got thrown away as soon as the publisher took on the book.

I did the math. I would need to sell about 10,000 books through that system before I see one dime.

I trust this editor. I assume she’s right, but by luck I happened to get into a conversation with an actual book publisher. This is a small house that only takes 70% of the money instead of the usual 85%. (If you self-publish through Amazon, they take 30%.) So I asked what magic formula they have that allows them to sell 233% more books than I would. “We have a newsletter.” Um, what? No, there must be more to it than that. I probably have more followers on this blog than that little publisher has subscribers to their newsletter.

They will make a cover (I have a cover, thank you very much), they will edit it (smart people have told me it doesn’t need any more editing), they will take care of copyright registration and ISBN (so will Amazon), and if they decide to do a print run (if they decide?), they have a deal with a big publishing house to get that done (Amazon has a subsidiary called “createspace” that will print on demand).

I’ve decided that the book publishing business in 2015 is basically the record industry of 1965. They screw the artists out of almost all the money, and produce a generic product they think will sell. They mostly just rely on a few hits and don’t make much on the bulk of what they do, so if you don’t happen to become a star, you’ll get nothing.

Homey don’t play that.

So I’m going to self-publish. So far I’ve spent $29 on stock photos for my cover (I’ll show you in an upcoming post), so by my math I’ll hit break even when I sell 9 books. Instead of 10,000. I’m confident I can sell 9 books. I’m confident a big publisher would not sell 10,000.

But to be completely fair, my manuscript did need to be edited. So I crowdsourced that. I sent it to about 30 people, and they gave me a huge range of feedback. They found lots of typos, and places where I used the same phrase an annoying number of times. They suggested places I needed a little more back story to motivate the characters. They suggested places I could “hang a lantern” to let the reader know I was doing something weird on purpose. They clued me in to things that might be hot-button issues for certain people, so I could decide whether to embrace or avoid those issues.

In the end, I added about another 4,000 words in response to all that feedback. That’s a lot, so I consider what my crowd did a pretty through edit.

The process—unlike what I think working with a regular editor would entail—was delightful. I learned that a lot of my friends (by which, I mean people I know from Twitter), read really fast. Laid out as a standard 5×8″ paperback, the book is just about 400 pages. A lot of people read that in a day or two. And a lot of people gave me real-time feedback as they were reading, which was a blast. I got to see the emotional impact I was having on my readers! It was what I imagine a film director feels when they go to the opening of their movie.

So crowdsource editing: highly recommend. Five out of five stars.

Everything I’ve read says that I’m supposed to start blogging about my book before it’s ready. And who am I to argue, since that’s basically blogging about my favorite subject: ME!

So you can expect that I’ll do more of that in the next few weeks as I get the final typos fixed, and figure out all the tech puzzles required to get it available on Amazon.

Question: Should I bother getting it onto anything other than print and Kindle? Does anyone actually buy books from the Apple iBooks store? What about the Nook? Is that still a thing?

It’s probably not that much work to get it onto those other platforms, but I don’t want to waste my time, either. Comments please!

Update: I made a cover and wrote a blurb.

Plagiarism, Twitter, and the DMCA

It’s been a pretty crazy weekend for the Plagiarism is Bad account. One of the collaborators (not me) noticed a strange thing when he did one of our usual searches to find people stealing a tweet. Some of the tweets were replaced with a notice that said they were being “withheld in response to a report from the copyright holder.” He tweeted about that (with a screen shot) and his tweet went viral, first via retweets and quotes, and subsequently on all the new media news sites.

So what’s going on? I’ve covered a lot of this before so I won’t get into the details here, but basically, Twitter couldn’t care less about tweet theft. They give blue “verified” checkmarks to accounts like “Men’s Humor” that contain nothing but stolen content and links to clickbait sites. They ignore the fact that one guy has created hundreds of accounts and populated them with stolen content and more clickbait links. Twitter really doesn’t care about this at all. They have a form you can use to report accounts that chronically steal, and as far as anyone knows, they’ve never acted on any report filed using that form.

However, there’s this funny law in the USA (and similar laws in other countries) that says unless you provide a way for copyright holders to request that copyright-infringing content is taken down, you the service provider become the copyright thief. The penalties for infringing copyright are draconian, and so it’s really important that if you run a site like Twitter that you follow these so-called “safe harbor” rules. In the USA, this law is called the DMCA.

The rules go like this: a copyright holder makes a complaint and the service provider gives the infringer a chance to counter. If the infringer says it isn’t stealing, then the service provider is off the hook, and the two parties can go fight it out in court. If the infringer ignores the report, then the service provider has to take the content down. The service provider does not need to weigh in on whether they think it actually is infringing content. It’s certainly safest for them if they just assume all allegations are legitimate.

So what’s happened is that at least one writer has gone ahead and asserted that they own the copyright on a joke. And Twitter simply treated this like any other DMCA “take down request” and took the content down. There’s some question whether they actually notified the person who did the theft. Their policy is that they will, but at least one admitted thief said they got no notice.

So does that mean that Twitter thinks jokes are subject to copyright? No, it really doesn’t. Twitter’s DMCA request form has some language that implies they think poems and song lyrics are. But whether Twitter itself thinks anything is subject to copyright is basically irrelevant. They are going to take down anything that gets reported because that’s the safest way to not end up in legal trouble. Twitter is in no way alone in this regard. Pretty much every service provider has that same policy.

So although it seems that Twitter has suddenly started doing something new, maybe they really haven’t. Maybe what’s new is that authors are using the DMCA form to report theft instead of the abuse form. And whereas I’m pretty sure the abuse form goes to a “write only database,” they actually have to read the DMCA reports or they can lose their “safe harbor” status and be held liable by litigious bulldogs like the record companies.

Whether jokes are subject to copyright is not settled law. I covered this in detail before, but the short version is maybe they are, maybe they aren’t. Making a DMCA complaint about a joke is perfectly reasonable, given how grey this is. As long as you really are the author, there’s nothing “in bad faith” about reporting the theft, and you aren’t going to get in any trouble for doing it, even if some day the courts decide once and for all that joke tweets aren’t protected.

But the DMCA is absolutely the wrong tool for this job. What Twitter should do is read the abuse reports, see that the accounts being reported are crap, and shut them off. Why they don’t is anyone’s guess. But I’m quite certain that would be a lot cheaper to do than to handle all this in the DMCA reporting system. Perhaps all the publicity being generated this weekend is going to cause a big surge in the DMCA requests and that might cause Twitter to take a step back and realize they can do both themselves and the community a favor by simply shutting down any account that steals tweets.

How to Write a Novel in 58 Days

Catchy headline, huh? Actually, I have no idea how you can write a novel in 58 days. But having just achieved that myself, I can tell you how I did it. I don’t know yet whether my novel is any good, so it’s possible that this is how you write a bad novel in that time period. Time will tell.

[Update: It’s got 4.8/5 stars on Amazon, so I guess people think it’s good.]

The novel is called “Entropy” and I think the genre is called “Erotic Romance Fiction.” I looked at a list of genres in Wikipedia and that one seemed like a really good fit. There’s a plot and characters and stuff, but there is also a lot of sex, and the sex is important to the plot. My first draft was about 81,000 words. When I started writing, I decided I wanted to write a novel, so I looked at articles on the web to find out how long a novel is. I found several sources that all had a pretty consistent answer: 80-90 thousand words.

So I set out to write 85,000 words figuring it would go up and down a bit after I finished. When things started to wrap themselves up at the end, I decided anything over 80K was fine.

When I started out, I only had a very basic idea: Middle-aged woman in a loveless marriage gets involved in an online romance. I figured that was a good thing to write about because judging from my Twitter feed, that’s a really common thing that happens. I wrote my opening scene, and then I fell into the writing process I’m about to describe. None of this was planned. It just happened this way.

I worked in “scenes” not chapters. I define a scene as a single place, group of characters, and plot point. I would think about what I want the next scene to be. Where should it happen? Who should be there? What’s the gist of it? That’s all I planned in advance. It turns out, that planning takes about a half a day. Sometimes in that planning, I’d come up with the next two scenes, but usually I’d just come up with one. I’d play with different ideas while I planned the scene: things people might say; key things that needed to happen. I would do all this planning while I walked the dog, or drove to work, or slept. I do a lot of my best thinking when I’m asleep.

Then I would look for a chance to steal an hour to write. I’d sit at my laptop and start typing. A lot of my scenes are dialog, and that was particularly easy. I just let my characters decide what to say. I guess it is kind of like improv theater. If you are true to the characters, you can just write what comes naturally. I would go back and forth, and see what happened. This is also the process I used when people were deciding what to do next. I just let my characters decide on their own.

Since I was completely out of the loop on decision making, I honestly could not possibly plan more than a scene in advance. I needed the characters to tell me how to move the plot forward. Then I would look at what happened, and start thinking about the next scene. There are 116 scenes in the book. That works out to exactly two per day. The average length of a scene is 700 words. That’s about how long a short blog article is. So you can see how banging that out in one sitting is not really a big deal. I always finished the scene in the same sitting I started.

There were some things I needed to research, and I did that in that between-typing time. But since this book is mostly about interpersonal relationships, there wasn’t too much research required.

I wrote using TextEdit on my mac, because it understands all the keyboard shortcuts from an old text editor called Emacs that I use in my job (I’m a computer programmer, mostly). I would write the draft in TextEdit, and then I’d email it to my cheerleader. My cheerleader is a person who would read my scene and tell me it was wonderful. This was key to my motivation. I was not looking for constructive feedback at this point. I was looking for someone to keep me going. And my cheerleader did a fantastic job of that.

TextEdit doesn’t tell you word count, but it saves files in a format called RTF that I could read with a little program I wrote to give me word count. So after I saved my scene, I’d run my program and it would tell me my new word count, and what percentage of my  85K word goal I’d reached. The percentage generally went up about 1% each time I saved a scene, so that was good motivation as well.

Periodically I would take all these scenes, each in its own RTF file, and copy/paste them into Microsoft Word. I hate MS Word. I find it impossible to compose in, but it’s good at managing books. I would proofread there, taking advantage of the green grammar squiggles and the red typo squiggles. Once I was comfortable that my doc was in good shape, I’d save a PDF and send it to my reader. This person’s job was supposed to be to help identify plot holes, raise questions I might answer going forward, etc. In fact, my reader turned out to mostly be another cheerleader.

A couple days after I started writing I had an idea for how to end the book. So I wrote that and set it aside. I had no idea if I’d end up using that ending, but it did give me a compass heading. If it worked out, I thought that would be a decent way to end. Other than that, I had no plot in mind when I started. Since I’ve finished the draft and given it to more people, several have told me they like the plot. So I guess there is a plot. I don’t see it. But they do, so shrug. I did have an existential crisis at about the 33% mark when I realized I was writing a book without a plot, and I had to decided whether that was okay. But I’ve read a lot of great books that had no plot, so what the hell.

But I guess there is a plot, and I know that I wrote a lot of great scenes that I really liked. I fell in love with my characters. They made me cry on more than one occasion, which frankly I find a little disturbing.

I’m not sure what happens next. I’ve given the draft to a few people for feedback, and I’ll tweak and tune the book to fix problems they find. And I’ve given it to a person in the book industry to help me figure out if it belongs in tradition publishing, or indie publishing, or self publishing. Once I figure that out, I’ll do another blog post, no doubt!

Update: I edited it.

Thoughts on the Plagiarism is Bad Project

A while back, myself and two collaborators started an account called @PlagiarismBad. (Background posts:  1 ,  2 ,  3 .) Things have evolved since we started it, and it seems like a good time to take a step back to see where we are.

What’s the Point?

This is a question I’m hearing both from my collaborators and from people we list. (Not so much from our fans.) I think there are three reasons for this project to continue: education, shaming, and inoculation. I’ll expand on each of these.

Education — A lot of people don’t understand that plagiarism is wrong, and they don’t understand in particular that copy/paste tweeting is plagiarism. So by shining a light on this practice, we are helping to end that ignorance. And it is clearly working. There have been many, many cases of people saying, “I’m sorry. I had no idea I shouldn’t do that. I’ll delete those posts!” And we take those people off our lists.

We are also helping to educate the general twitter population about how incredibly widespread the plagiarism problem is on twitter. There are more than 4,700 people on the main tweet thief list now. Twitter lists only allow a maximum of 5,000 names, so we will have to add a second list soon.

Shaming — Initially, the main point of the project was to shame tweet thieves into stopping this particular behavior. In a social network where “Reporting” people apparently has no impact whatsoever, this is probably the only way to combat the problem. By putting people on a list, we embarrass them. And we put everyone else on notice that certain people are thieves, so those other people may shun them.

There is plenty of evidence that this works. People react strongly to being listed. They argue they are not thieves. When we show them evidence, they either slink away, or switch to the “so what? who made you boss?” argument. The only people who seem to think what we are doing is a waste of time are the tweet thieves. The people who create original tweets seem to really appreciate what we are doing.

Inoculation — I’ve created an app that lets people easily block everyone on the tweet thief list. It also can periodically update those blocks so that as new thieves are added, they will be blocked as well. This effectively inoculates people from having to look at stolen tweets. That was one of my main personal objectives of the project in the first place. I found it embarrassing that I was following thieves and favoriting and even retweeting their stolen tweets. With the automatic blocking, I really don’t have to worry about that any more.

People may think that it also protects their tweets from being stolen. Unfortunately, this is not really the case. There are any number of ways that a thief you have blocked might see your tweet. They might see it being sent by another thief. Or they might pick it up from your FavStar page. Or perhaps your tweet was so good that it made it into a stolen-tweet-compilation site. (Yes, these exist. There are several. And most thieves believe that copying from these sites is not plagiarism, in the same way that ripping off a drug dealer isn’t a crime, I guess.)

Twitter’s Terms of Service

It is quite clear that Twitter (the company) could not care less about tweet theft. You can report people, and Twitter will do nothing. There are massive, verified (“blue check”) accounts that do nothing but post stolen tweets. There are so many that we actually made a list of those accounts (and a handful of other accounts that people are invariably shocked to learn are chronic thieves). Nonetheless, there are two places in Twitter’s terms of service where tweet theft is forbidden: the copyright rules, and the spam rules.

Twitter’s terms forbid you from posting content to which someone else owns the copyright. Whether tweets of words fall under this rule is a matter of debate, which I will cover later. However, most of the pictures you see on twitter (except selfies and foodies) are protected by copyright, and those are pretty much never owned by the person posting them. If you removed all the copyright violating pictures from twitter, you’d basically have no pictures on twitter (except pictures of faces, flesh, and flan). So it is pretty obvious that twitter does not take this service term seriously. But it’s in there.

The spam rules are written in a fluid way, listing many things that might make twitter consider you a spam account. One of the things explicitly listed is tweeting other people’s tweets and pretending that you wrote them. This is the key reason my collaborators and I believe that tweet theft violates Twitter’s terms of service. Because the terms of service says it does. It’s a pretty compelling argument. However, as with copyright violation, there is no evidence of Twitter caring the least bit about enforcing this either.

Twitter only seems to deactivate accounts that follow/unfollow too fast. Other than that, it certainly appears that they completely ignore their own terms of service. I wonder whether Twitter’s legal counsel knows this. It seems unwise, but I’m not a lawyer.

Copyright vs. Plagiarism

A lot of people get wrapped around the axle of whether plagiarism is illegal under copyright law. But really, it doesn’t matter. Plagiarism is wrong. It is unethical. It is immoral. Taking someone else’s words and passing them off as your own is stealing. You shouldn’t do it. It doesn’t matter whether it is illegal. It is wrong regardless of whether it is legal or not.

Okay, so that said, is it illegal? As with many legal issues in intellectual property law, the answer is a mix of “it depends” and “nobody really knows, because it hasn’t been to court.” For plagiarism of tweets to be illegal under copyright law, two things would need to be true. The tweets would have to be copyrightable, and the copying would have to not be “fair use.”

Some tweets, like “Damn it’s cold” are not entitled to copyright protection under the law. Some tweets, such as short poems (Haiku, Senryu, “six words”, etc.) probably are entitled to copyright protection. Songs are entitled to protection, and 140 characters is probably enough to convey a melody in solfège. A single joke tweet, which is mostly what people plagiarize is probably not. However, a whole timeline of jokes certainly is. And we have seen accounts that do exactly that: tweet everything another account has ever said. That’s clearly a copyright violation in the USA. (“In the USA” is an important qualification, because every country has different rules and judicial precedents about copyright law.)

So the vast majority of the plagiarism we flag on the account is not copyright violation, because the tweets cannot be copyrighted. But it is still plagiarism. And that’s the whole point.

The account isn’t called “Copyright Violation is Bad.” It is called “Plagiarism is Bad.”

And so…

The fight continues. My collaborators and I (and just to be clear, they are doing all the work; I just set the thing up and wax poetic about it here on my blog), are going to keep educating, shaming, and inoculating. And soon the list of tweet thieves will hit 5000 people and we will have to start a second list. Sigh.

Tweet Detective

The @PlagiarismBad project has been moving along nicely. I wrote about it here and here. As I write this we have 1,636 tweet thieves on the list. That means that my collaborators and I have manually listed all those people after seeing firsthand evidence that they blatantly copied a tweet.

And we aren’t talking about tweets that are just similar. We are talking about tweets that are exactly the same. Genuine copy and paste tweets.

There are two ways to find a thief. The easy way is to start with a great tweet and search for it using the “Search” field in Twitter. You can then look at each user who copied it and failed to give any indication that it was not their original idea. No quotes. No “MT” or “RT”. No credit. Those people get added to the list. The only tricky part is that you can only list about 100 people an hour, and then Twitter puts the brakes on.

The hard way is to start with a suspect. Typically this is someone that was reported to us as a thief. The process of confirming these allegations is pretty labor intensive. You need to manually copy and paste each of their tweets into the Twitter Search box and then pore through the results, looking for a match. This looked to me like something that could use some automation. So, being me, I wrote a web app.

You give the app the handle of a suspect, and it finds up to 50 recent tweets by that person that are fairly long, don’t contain any links, don’t have “MT” or “RT” in them, and are not @-replies. It then uses Twitter’s “Search API” (which is not the same as the “Search” box in Twitter) to look for earlier tweets that are similar. It goes through the results if there are any, and reports the oldest, bestest match it can find.

There are a few problems with this app, though. Twitter’s API for getting a user’s tweets is basically like going to their timeline and scrolling down. You get all the @’s and retweets mixed in. So if the person you are looking at does a lot of that, it can take quite a while to get good list of tweets to search.

Also, Twitter has a really low rate limit of 180 searches every 15 minutes, which means you can only look up a few people before you bump into that limit.

But the biggest issue is that the Twitter Search API is horrible. It doesn’t return anything but really recent tweets. You can give exactly the same query to the search box on twitter.com and get hundreds of matches, and the Search API reports no results at all! (Programs cannot use the good search on twitter.com, so we have to use the Search API.) It’s a mess, and the developer support forums are full of people basically saying, “What the fuck?” and the Twitter support people replying, “Yeah, sorry about that.”

It turns out that Twitter has no intention of fixing this. They bought a company last year that provides good historical search results. If Twitter makes good historical data available in the Search API, people wouldn’t have to pay for that service any more.

What this means is that a really horrible tweet thief who copies tweets right away and does it all the time is easily outed by the app. But a person who just every now and then pretends to write something they originally saw on someone’s FavStar page or in an e-Card is not likely to show any results.

If you want to try it, go to tweetdetective.appspot.com and log in with your Twitter account (the program runs the searches as you). Then put in a suspect’s Twitter handle and see what you find. Have fun!

No Rest for Merry Gentlemen

A couple weeks ago I took you through the song We Three Kings, played five completely different ways. That was kind of fun, so I thought maybe I’d do that once more. This time, we’ll be looking at the song God Rest Ye Merry Gentlemen. As before, let’s start with my brother’s version as our point of reference. You can buy the tracks here. (Scroll to the bottom of the page.)

The song opens with a very straight, classic version of the melody. Next, the tuba comes in with a walking bass line. That means he is “walking” up and down the scales and arpeggios that go with the chord progression of the song. You often hear bass players do this (particularly upright double-bass players) in jazz. It’s much less common to hear a tuba doing a walking bass line, because it’s incredibly hard to play tuba continuously like that. Most players just don’t have the chops.

The second time through the melody, and you can hear they are swinging it now. Swinging is another jazz thing. When you play straight time, you divide each beat into two parts. But when you swing, you divide each into three parts and play the first and third (skipping the second). You can take any song, swing it, and it will sound a lot like jazz.

So that’s our reference point. Basically the melody is played three times, in a slightly different way each time.

Next up is a version from Roy Hargrove, on an absolutely excellent jazz Christmas album called A Merry Jazzmas. You can get the album here on iTunes.

If you don’t know Roy Hargrove, you need to check him out. He’s the trumpet player here, and I think he’s basically the Lee Morgan of our generation. Like Lee, he was discovered at an early age by another famous player. (Lee was discovered by Dizzy; Roy was discovered by Wynton.) They both had a lot of success while young. (Lee is the trumpet player on Coltrane’s Blue Trane, at age 19; Roy’s first solo album was released when he was just 21.)

The structure here is a classic head/solos/head (I cut off my sample partway through the solos, in deference to the musician’s copyrights, of course). The head is the melody of the piece. In most jazz music, you play that once or twice, then people solo for a while, and then you play the melody one more time. When you are watching live players soloing and you see one of the players point to his head, he means “Okay, that’s enough solos, let’s go back and play the head one more time.” Often the solos are played over the chord changes of the melody, but in this case they switch to a different progression.

Next up we have Loren Schoenberg on A Jazz Christmas: Hot Jazz for a Cool Night. Here’s the full album on iTunes.

Loren is a great tenor player but is actually more famous for writing liner notes. He even won a Grammy for them once. This version follows that same head/solos/head form, but here we have the chord progression (what we jazz musicians call “changes”) of the melody for the solos. These are more challenging to solo over than the standard progression Roy used in his version, and the guitar player (not sure who that is) is really struggling to come up with something lyric. Dude. Been there.

Next up, let’s hit a version from one of my very favorite holiday jazz albums, Merry Magic by Eric Reed. As usual, here’s the link to the full album on iTunes.

Eric is a piano player, but the instrument you hear primarily here is a vibraphone, played by Steve Nelson. The “vibes” are basically like a xylophone, except the bars you hit with mallets are made out of metal instead of wood. There are metal tubes of varying lengths suspended below the metal bars that act as a resonance chamber for each (lower notes have longer tubes, because they have longer wavelengths and hence need more distance to resonate). Inside each tube there is a baffle on a rod that is spun by a motor at the end of the vibraphone. Usually, there is a knob you can turn to adjust the speed. The baffle opens and closes the tube, giving a vibrato to the sound coming out. Hence the name vibraphone.

Before we leave that version, check out what happens at the end:

After a remarkably long pause, they start playing a little round (that’s where multiple instruments are playing the same thing, starting at different times). And then it devolves into what we call a “vamp.” That’s where the musicians are basically fooling around on a simple chord repetition. Like trading fours, except less structured than that. They take turns, play over each other, listen and respond.

So at the start of that little clip we had the bass playing the melody. Suppose you do that as the head. Just have the bass play the melody and have some other instruments play crazy shit that is totally orthogonal. Would that be a good idea? Wynton thought so on Crescent City Christmas Card. Here’s the full album on iTunes.

Again, we have the head/solo/head structure. But the head and the solos have basically nothing to do with each other. The tempo, changes, everything are different. (And thank goodness, because although that bass melody was interesting, that tempo and structure would have been unlistenable for the whole song.)

So as your parting gift, I’ll give you a great version from Straight No Chaser. They switch up the time signature, doing this one in three instead of four like everyone else. Even though this is a vocal version, they still do the head/solo/head thing, although they keep the solo section really short.