Which hunting on Grammarly

It’s that time again! Time to see what’s going on in Crazy Grammar Town! Let’s visit our old friends, Grammarly. They have a post on relative clauses which is not bad, until they get to talking about that and which (yeah, I know this is more about style than grammar, just stay with me):

Confusion about when to use that and which has arisen for good reason: British and American English have different rules for them. In American English, that is used to introduce restrictive clauses, and which introduces nonrestrictive clauses.

The lamp, which was given to me by Aunt Betsy, is on the bedside table.

The lamp that Aunt Betsy gave me is on the bedside table.

In British English, it is often acceptable to substitute which in restrictive clauses.

The lamp which Aunt Betsy gave me is on the bedside table.

Of course, that could also be used acceptably in British English, which makes it safer, by default, to follow the American rule when in doubt. It also makes it easier to decide whether to insert commas, because if you follow the American rules, you can remember that commas should not precede that, but they should precede which.

Sounds easy, right? Sure… But you’re probably wondering if Grammarly, a company based in San Francisco, US of A, follows their own advice. Hahahahahahahahahahahahahahaha. Of course, they don’t, silly.

In their post on nouns, there are four instances of the word which. How many of these appear in restrictive clauses – the kind of clause that Grammarly specifically states should be introduced by that? Let’s count them:

One…

Grammarly which 1

Two…

Grammarly which 2

Three…

Grammarly which 4

Four!

Grammarly which 3

That’s all four! Congratulations, Grammarly! You hit for the cycle! *Applause*

Why is Grammarly so bad at this? Grammarly can’t follow their own advice because either:

  1. They are an editing company that apparently can’t even edit their own writing
  2. Their claim about US English is a hot pile of garbage
  3. They prefer you’d do as they say and not as they do
  4. All of the above

I’ll take Door #4, Monty.

Read more about which hunting and the that/which distinction from Jonathon Owen here and Stan Carey here.

Advertisements

Grammarly also sucks at subjects

Hey, remember when I wrote about the YUNiversity and their crazy ideas on what a subject is? Remember how they said the subject of sentence is ALWAYS a noun? Well, they’re not the only ones in crazy grammar town. Grammarly also likes to play fast and loose with grammatical subjects. Check it:

Grammarly subjects

First off, not every sentence needs a subject. Most do, but not all. For example, imperative sentences do not have subjects because the subject is often implied:

Just do it.

Don’t worry, be happy.

Get to the choppaaaaaaaa!

What’s a little crazy is that Grammarly uses an imperative sentence as an example to show that nouns can act as objects. Do they think that Give is the subject of their example sentence? (Narrator’s voice: It’s not. The sentence doesn’t have a subject.)

Grammarly imperative object

Second, like I said in the YUNiversity post, the subject of a sentence in English is not always a noun. It often is, but not always. The following can act as the subject in English:

Dummy it – It’s hot.

Unstressed/existential there – There’s plenty of time.

Prepositional phrase – Up in the front will suit me fine.

Adverb phrase – Gently does it.

Adjective phrase – The comic told some funny jokes.

All types of clauses – That he failed his driving test surprised everybody.; What Grammarly wrote online shocked me.

And more!

The rest of the info on the Grammarly blog post is pretty good, so that’s nice. I don’t know if Grammarly is copying stuff off of YUNiversity or if it’s the other way around. But somebody is cheating off somebody.

Poetry and Prose, Computers and Code

Back in February, I analyzed WordPress’s automated grammar checker, After the Deadline, by running some famous and well-regarded pieces of prose through it. I found the program lacking. What I wrote was:

If you have understood this article so far, you already know more about writing than After the Deadline. It will not improve your writing. It will most likely make it worse. Contrary to what is claimed on its homepage, you will not write better and you will spend more time editing.

I think my test of After the Deadline proved its inefficiency, especially since I noticed that the program finds errors in its own suggestions. Talk about needing to heed your own advice…

A comment by one of the program’s developers, Raphael Mudge, however, got me thinking about what benefit (if any) automatic grammar checkers can offer. Mr. Mudge noted that the program was written for bloggers so running famous prose through it was not fair. He is right about that, but as I replied, the problem with automated grammar checkers really lies with the confidence and capability of writers who use them:

[The effect that computer grammar checkers could have on uncertain writers] is even more important when we think of running After the Deadline against a random sample of blog posts, as you suggest. While that would be more fair than what I did, it wouldn’t necessarily tell us anything. What’s needed is a second step of deciding which editing suggestions will be accepted. If we accept only the correct suggestions, we assume an extremely capable author who is therefore not in need of the program. As the threshold for our accepted suggestions lowers, however, we will begin to see a muddying of the waters – the more poorly written posts will be made better, but the more well written posts will be made worse. The question then becomes where do we draw the line on acceptions to ensure that the program is not doing more harm than good? That will decide the program’s worth, in my opinion.

As it turns out, after that review of After the Deadline, I was contacted by someone from Grammarly, another automated grammar checker. For some reason they wanted me to review their program. I said sure, I’d love to, and then I promptly did nothing. In truth, I was sidetracked by other things – kids, work, beer, school, the NHL playoffs, more beer, and recycling. So much for that.

Now R.L.G. over at the Economist’s Johnson blog has a post about these programs and a short discussion of Ben Yagoda’s review of Grammarly at Lingua Franca, a Chronicle of Higher Education blog. I want to quickly review these posts and add to my thoughts about these programs.

First, R.L.G. rightly points out that “computers can be very good at parsing natural language, finding determiners and noun phrases and verb phrases and organising them into trees.” I’m happy to agree with that. Part-of-speech taggers alone are amazing and they open up new ways of researching language. But, as he again rightly points out, “Online grammar coaches and style checkers will be snake oil for some time, precisely due to some of the things that separate formal and natural languages.”

Second, Mr. Yagoda’s review of Grammarly is spot on. (I’m impressed by how much he was able to do with only a five day trial. They gave me three months, Ben. Have your people call mine.) Not to take anything away from Mr. Yagoda, but reviewing these checkers is like shooting fish in a barrel because they’re pretty awful. A rudimentary understanding of writing is enough to protect you from their “corrections”. But it’s the lofty claims of these programs that makes testing them irresistible to people like Mr. Yagoda and myself.

So who uses automated grammar checkers and who could possibly benefit from them? The answer takes us back to the confidence of writers. Obviously, writers like RLG and Ben Yagoda are out of the question. As I noted in my comment to Mr. Mudge, the developer of After the Deadline, “a confident writer doesn’t need computer grammar checkers for a variety of reasons, so it’s the uncertain writers that matter. They may have perfect grammar, but be lead astray by a computer grammar checker.” It’s even worse if we take into account Mr. Yagoda’s point that “when it comes to computer programs evaluating prose, the cards never tell the truth.”

We do not have computers that can edit prose, not even close. What we have right now are inadequate grammar checkers that may be doing more harm than good since the suggestions they make are either useless or flat out wrong. They are also being peddled to writers who may not be able to recognize how bad they are. So there’s a danger that competent but insecure writers will follow the program’s misguided attempts to improve their prose.

It’s strange that Grammarly would ask Mr. Yagoda or myself to review their program since Mr. Yagoda is clearly immune to the program’s snake oil charm and I wasn’t exactly kind to After the Deadline. But such bad business decisions might prove helpful for everyone. Respected writers will point out the inadequacy of these automatic grammar checkers, which will hopefully influence people to not use them. At the same time, until these programs can really prove their worth – or at least not make their inadequacy so glaringly obvious – they will not receive any good press from those who know how to write (nor will they get any from lowly bloggers like myself). In this case, any press is not good press since anyone reading R.L.G. or Ben Yagoda’s discussion of automated grammar checkers is unlikely to use one, especially if they have to pay for it.

[Update – Aug. 9, 2012] R.L.G. at Johnson, the Economist’s language blog that I linked to above, heard from Grammarly’s chief executive about what the program was meant for (“to proofread mainstream text like student papers, cover letters and proposals”). So he decided to put Grammarly through some more tests. Want to guess how it did? Check it.