Serenity Software
Editor: Software for Writers and Teachers
what's new      about      * trial *      FAQ      buy now      upgrades      contact us      home     

Comparing Grammar Checkers:

Holding Grammar Scammers' Feats to the Fire

                                "Corect all your gramar errors and enhanse your writing."

                          This 2013-2014 advertisement for Grammarly has no grammatical
                            mistakes to correct, and its three spelling mistakes are easily
                              caught by your word processor.  What is it they are sellimg?

Grammar checkers deliver much less than they promise.  Most promise to improve customers' writing dramatically, even to "perfect" it.  But even near-perfection is logically and linguistically impossible.  The number of contextual-spelling mistakes possible in English that spelling checkers cannot catch is uncountable, for example.  So are the number of possible compound modifiers needing hyphenation and the number of possible wordy phrases.  No computer program can find and correct all such problems, because there are *no general rules* for identifying them.

Writers are pleased when a grammar checker finds mistakes in their work, and most checkers do that.  But what writers don’t know is how often the checker (1) overlooks common or obvious mistakes, (2) falsely flags correct expressions as mistakes, and (3) offers, as "corrections," new mistakes.  The only way to discover that information and answer these three crucial questions is by direct comparison of their performances.

Below are results of a comparative-performance study of 20 grammar checkers, including a leading Mac checker.  The study uses real-world examples of grammatical mistakes and stylistic problems taken from published sources.  The 8 test documents include 745 mistakes and problems in 5 categories of grammar and 3 categories of style.  A brief description of the test precedes each list of results.  (The 9th test is of how well a checker handles differences between US and UK English and can help writers of both.)

Programs tested are AutoCrit, After the Deadline, ClearEdits, Correct English, Editor, Ginger, GrammarBase, GrammarCheck.net, Grammar Check Anywhere, Grammar Expert Plus, Grammarian Pro (for the Mac), Grammarly, Grammar Slammer, PaperRater, Pro Writing Aid, SpellCheckPlus, StyleWriter, and WhiteSmoke.  Results for Word’s and WordPerfect's onboard grammar and spelling checkers are included for comparison.  The study was carried out during late 2012 and early 2013 using free-trial versions of the programs.  (Grammarian Pro was tested by Mac expert Mike Wallbridge in late 2013.)

At the end of these results, under "Other Checking Programs," we offer information and comments on another 20 or so checking programs that, for one reason or another (we give the reasons), we decided not to include in our main study.

Click a test category for a brief description of the test and to see the results.

Grammar: contextual spelling and usage
    (click link to close)

      The test: An essay by a prominent authority on language, who deliberately included 63 common mistakes in
      contextual spelling and usage.   Examples: beckon call, piece of mind, all tolled.    
  Results out of  63   mistakes:                   

After the Deadline 15 9 1   
AutoCrit 23 0 1   flags ordinary, but no contextual, misspellings
ClearEdits 0 1 3   found no misspellings or misusages
Correct English 18 14 8   
Editor 31 0 0    (57 now with "brute force" additions: see "No General Rules," above)
Ginger 15 9 1   
GrammarBase 17 12 4   
GrammarCheck.net 14 8 3   finds no contextual mistakes
Grammar Check Anywhere 17 7 7   flags ordinary, but no contextual, misspellings; 4/5 homonyms wrong
Grammar Expert Plus 14 9 2   
Grammarian Pro (Mac) 18 1 3   test results courtesy of Mike Wallbridge
Grammarly 17 11 2   few contextual mistakes found; we added confusing advice to blue #
Grammar Slammer 20 7 3   only 1 contextual spelling mistake found
PaperRater 18 9 8   removes apostrophes and dashes, then says results are mistakes
Pro Writing Aid 16 10 1   
SpellCheckPlus 23 10 4   
StyleWriter 14 13 7   flags ordinary, but no contextual, misspellings
WhiteSmoke 24 6 7   
Word 17 9 4   
WordPerfect 15 15 4   
Grammar: hyphenation and compounding
    (click link to close)

      The test: A set of sentences containing 100 examples, from a standard US writer's reference manual,
      of hyphenation and compounding mistakes--"probably the most common spelling questions for writers
        and editors" (Chicago Manual of Style)
.   Examples: blue green algae, highly-paid, stick in the mud.    
  Results out of  100   mistakes:                   

After the Deadline 23 0 10   
AutoCrit 0 0 0   no help with these common problems
ClearEdits 0 0 1   no help with these common problems
Correct English 13 6 23   
Editor 87 0 7   
Ginger 4 0 7   
GrammarBase 27 2 9   
GrammarCheck.net 25 2 8   
Grammar Check Anywhere 4 10 16   ignorance of hyphenation causes many syntax errors
Grammar Expert Plus 3 3 15   misses most required hyphens; many false corrections
Grammarian Pro (Mac) 18 1 3   test results courtesy of Mike Wallbridge
Grammarly 4 1 12   failure to recognize misspellings caused 8 of the false corrections
Grammar Slammer 2 3 6   
PaperRater 1 1 3   
Pro Writing Aid 26 1 6   says spelling and hyphenation are "consistent" but misses many errors
SpellCheckPlus 9 3 14   
StyleWriter 37 4 4   
WhiteSmoke 40 5 7   
Word 22 1 2   
WordPerfect 16 10 9   
Grammar: punctuation, spacing, mechanics
     (click link to close)

      The test: A made-up text containing 42 deliberate mistakes in spacing, punctuation, and quotation.  Some
      are typing errors, the rest are errors according to modern US copyediting rules (UK style differs in 3 instances).
      Examples: "text -checking" "happen,of course" "(p. 345)."    
  Results out of  42   mistakes:                   

After the Deadline 2 1 2   
AutoCrit 0 0 0   
ClearEdits 0 0 3   does not check punctuation or spacing
Correct English 19 2 5   
Editor 36 0 0   we made up the test; Editor already knew most of the standard rules
Ginger 4 5 4   baffled by ellipses
GrammarBase 2 1 1   
GrammarCheck.net 2 0 1   
Grammar Check Anywhere 4 0 18   baffled by ellipses, wants new sentences
Grammar Expert Plus 6 1 14   baffled by ellipses
Grammarian Pro (Mac) 23 4 0   test results courtesy of Mike Wallbridge
Grammarly 7 7 10   baffled by ellipses
Grammar Slammer 9 3 10   baffled by ellipses
PaperRater 0 3 1   
Pro Writing Aid 0 3 0   
SpellCheckPlus 10 6 6   
StyleWriter 10 5 2   baffled by ellipses and loose hyphens
WhiteSmoke 12 0 11   baffled by ellipses
Word 12 2 0   
WordPerfect 24 1 8   excellent at finding and fixing ellipsis errors
Grammar: possessives and plurals
 (click link to close)

      The test: A set of sentences, from the news and other print sources, containing 41 possessive-plural mistakes
      (in Chicago style)—a problem for many writers.  The sentences include their-there and it's-its confusions, two
      verbs and some plural nouns misspelled with 's and s', and an our-are homonym error.   Examples: Its a shame,
      ones' money, there own way.    
  Results out of  41   mistakes:                   

After the Deadline 6 0 1   5 out of the 6 mistakes identified were its or it's
AutoCrit 5 0 0   marked 5 mistakes as "overused words"
ClearEdits 0 0 0   found one unnecessary word
Correct English 19 4 5   
Editor 22 5 2   
Ginger 9 5 1   
GrammarBase 7 1 2   
GrammarCheck.net 6 0 2   
Grammar Check Anywhere 3 1 2   found 3/5 their/there; can't tell its from it's, possessives from plurals
Grammar Expert Plus 11 8 3   
Grammarian Pro (Mac) 7 0 1   test results courtesy of Mike Wallbridge
Grammarly 13 7 11   most "enhancement" suggestions are mistakes
Grammar Slammer 10 2 1   
PaperRater 6 0 0   
Pro Writing Aid 7 0 0   
SpellCheckPlus 20 7 2   
StyleWriter 1 8 1   misses all 5 there/their mistakes; flags all its/it's, correct or not
WhiteSmoke 15 4 1   
Word 19 3 1   
WordPerfect 15 10 3   
Grammar: problems in sentences
(click link to close)

      The test: Ninety sentences from published sources, building on a long list compiled by a noted critic of bad
      writing.  The sentences include a wide range of common writing problems—in subject-verb agreement, verb
      tenses, modifiers, usage, phrase structure, sentence fragments, punctuation, ESL wording, and more.  Many
      test sentences are short, which should make their bad grammar easier for software to recognize.  The 90
      sentences have 120 mistakes in grammar, usage, punctuation, and spelling among them.

      NOTE: This is the broadest and most comprehensive of our comparisons.  We expected grammar checkers to do
      poorly on this test, despite their claims, because of the complexity of English at the sentence level.  The results
      confirm our expectations.   Examples: to whom you were speaking to, it started off routine, balls of slime is eaten.    
  Results out of  120   mistakes:                   

After the Deadline 5 4 7   found 1/36 verb mistakes, 0/3 sentence fragments
AutoCrit 0 0 0   found no grammatical problems
ClearEdits 6 2 2   found 1/36 verb mistakes, 0/3 sentence fragments
Correct English 36 4 9   found 14/36 verb mistakes, 1/3 sentence fragments
Editor 55 4 7   found 5/36 verb mistakes, 2/3 sentence fragments
Ginger 17 4 4   found 8/36 verb mistakes, 0/3 sentence fragments
GrammarBase 8 3 8   found 2/36 verb mistakes, 0/3 sentence fragments
GrammarCheck.net 8 3 8   found 2/36 verb mistakes, 0/3 sentence fragments
Grammar Check Anywhere 19 5 18   found 6/36 verb mistakes, 0/3 sentence fragments
Grammar Expert Plus 16 10 10   found 7/36 verb mistakes, 0/3 sentence fragments
Grammarian Pro (Mac) 39 1 21   results courtesy of Mike Wallbridge; 11/36 verb mistakess, 0/3 frag
Grammarly 33 9 13   found 15/36 verb mistakes, 1/3 sent frag, wrongly ID'd 6 others as frag
Grammar Slammer 14 2 2   found 2/36 verb mistakes, 0/3 sent frags; flags all proper names
PaperRater 8 2 3   found 3/36 verb mistakes, 0/3 sentence fragments
Pro Writing Aid 7 3 4   found 2/36 verb mistakes, 0/3 sentence fragments
SpellCheckPlus 20 5 12   found 9/36 verb mistakes, 0/3 sentence fragments
StyleWriter 17 5 6   found 2/36 verb mistakes, 0/3 sentence fragments
WhiteSmoke 28 3 19   found 17/36 verb mistakes, 1/3 sentence fragments
Word 27 8 5   found 14/36 verb mistakes, 0/3 sentence fragments
WordPerfect 34 9 4   found 13/36 verb mistakes, 1/3 sentence fragments
Style: redundancy and repetition
(click link to close)

      The test: An essay by a popular author of books on language, playfully written to highlight many common but
      unnecessary repetitions in published work.  The essay has at least 192 redundancies, tautologies, and other
      needless terms.  Counts include items flagged variously as clichés, "complex terms," and redundancies.
      This category is not the same as wordy phrases.  Examples: past history, ATM machine, sum total.    
  Results out of  192   problems:                   

After the Deadline 34 2 16   
AutoCrit 12 0 2   all identified as clichés; flags no repetitions or redundancies as such
ClearEdits 27 4 3   
Correct English 46 2 15   has trouble with singular/plural verbs
Editor 142 0 8   
Ginger 1 3 22   stumbles over internal punctuation, foreign terms
GrammarBase 40 2 18   "111 critical mistakes was found in you writing" [sic]
GrammarCheck.net 34 1 13   
Grammar Check Anywhere 24 5 14   plus 14 homonym suggestions, all wrong
Grammar Expert Plus 23 3 11   
Grammarian Pro (Mac) 22 0 1   test results courtesy of Mike Wallbridge
Grammarly 35 1 27   also rejects 13 personal pronouns as inappropriate in "General" style
Grammar Slammer 22 0 12   
PaperRater 31 1 10   
Pro Writing Aid 50 1 0   claims to find 75 but flags only 50
SpellCheckPlus 10 1 16   good explanations of proper corrections
StyleWriter 55 11 9   
WhiteSmoke 25 6 22   two redundancies "corrected" by new redundancies
Word 14 1 8   
WordPerfect 0 3 3   
Style: clichés and overused expressions
  (click link to close)

      The test: Three passages, from different sources, with a total of 72 clichés and overused expressions.  Sources
      are a book on improving writing, a university writing-center workshop exercise, and a random-cliché picker on
      the Internet.   Examples: cute as a button, finer things in life, sadder but wiser.    
  Results out of  72   problems:                   

After the Deadline 8 0 2   
AutoCrit 15 0 0   
ClearEdits 1 0 0   no significant list of clichés or overused expressions
Correct English 8 0 2   
Editor 52 0 3   
Ginger 0 0 1   finds no clichés or overused expressions
GrammarBase 9 0 2   
GrammarCheck.net 9 0 2   
Grammar Check Anywhere 2 0 4   claims to check for clichés
Grammar Expert Plus 2 0 0   no significant list of clichés or overused expressions
Grammarian Pro (Mac) 4 0 0   test results courtesy of Mike Wallbridge
Grammarly 0 0 13   finds no clichés or overused expressions, says "clichés" is misspelled
Grammar Slammer 2 0 0   dislikes preferred US spelling of cliché
PaperRater 10 0 2   
Pro Writing Aid 24 0 3   
SpellCheckPlus 1 0 4   no significant list of clichés or overused expressions
StyleWriter 10 0 5   
WhiteSmoke 9 0 9   
Word 4 0 4   no significant list of clichés or overused expressions
WordPerfect 0 0 3   not a category that WordPerfect checks for
Style: wordy phrases

  (click link to close)

      The test: A set of 74 sentences with at least 115 wordy or unnecessary expressions, compiled from several college
      writing-instruction labs and a writer's blog.  Some sentences are short, a few are long, and all contain common
      wordy terms.  This category is not the same as redundancy and repetition, although both categories characterize
      sloppy writing.   Examples: due to the fact that, in advance of, have been shown to be.    
  Results out of  115   problems:                  

After the Deadline 11 6 5   
AutoCrit 5 0 1   wordy phrases are called "clichés"; "no redundancies" found
ClearEdits 35 16 4   many real problems flagged but incompletely or wrongly corrected
Correct English 36 0 6   
Editor 75 3 2   
Ginger 0 0 5   not a category that Ginger checks for
GrammarBase 18 5 3   
GrammarCheck.net 12 7 3   
Grammar Check Anywhere 41 4 16   syntax problems
Grammar Expert Plus 41 3 10   
Grammarian Pro (Mac) 36 0 1   test results courtesy of Mike Wallbridge
Grammarly 5 1 4   
Grammar Slammer 36 0 6   
PaperRater 1 0 1   not a category that Paper Rater checks for
Pro Writing Aid 2 0 0   no check for wordiness; both items counted are clichés
SpellCheckPlus 1 0 7   not a category that SpellCheckPlus checks for
StyleWriter 58 6 3   
WhiteSmoke 16 0 16   many "corrections" are new mistakes
Word 12 1 6   
WordPerfect 0 0 5   not a category that WordPerfect checks for
Style: American and British English
(click link to close)

      The test: A set of 39 pairs of sentences that alternate between common expressions in UK English and their
      counterparts in US English.  The expressions come from Internet sites where US-UK differences are discussed
      and illustrated.  About a third of the differences are in spelling and punctuation, the rest are in usage.
      None of the sentences contain grammatical errors.  The test's purpose is to discover whether the grammar
      checkers can recognize differences between major dialects of English and notify writers accordingly.  We also
      looked for user settings that specify which dialect to accept as standard while checking.  Examples: favourite /
      favorite, lorry / truck, at university / in college.     
  Results out of  39   problems:                  

After the Deadline 0 0 N   no US <-> UK help; UK spelling treated as mistakes
AutoCrit 0 0 N   ignores UK spelling, punctuation, usage
ClearEdits 1 0 ?Y   US<->UK setting makes almost no difference
Correct English 12 0 ?Y   spelling & punctuation only; settings option did not work
Editor 23 26 Y   2 false corrections
Ginger 7 0 N   only UK->US; 5 mistakes; US abbreviations taken as sentence endings
GrammarBase 0 0 N   no US <-> UK help; UK spelling treated as mistakes
GrammarCheck.net 0 0 N   no US <-> UK help; UK spelling treated as mistakes
Grammar Check Anywhere 7 0 ?N   "multilingual" checker, but trial has no US<->UK switch; 6 false corrs.
Grammar Expert Plus 5 0 N   users must provide own US->UK dictionaries, if they want them
Grammarian Pro (Mac) 4 1 Y   results courtesy of Mike Wallbridge; 3 false corrs.
Grammarly 8 0 ?Y   changed checking language to UK, made no difference; 10 false corrs.
Grammar Slammer 8 0 N   UK spelling and punctuation treated as mistakes
PaperRater 0 0 N   ignores UK spelling, punctuation, usage
Pro Writing Aid 0 0 N   ignores UK spelling, punctuation, usage
SpellCheckPlus 2 0 N   some UK terms "corrected" as mistakes; 9 false corrections
StyleWriter 10 12 Y   1 confusion, 1 false correction; almost no punctuation help
WhiteSmoke 3 0 N   9 false corrections, many nonsensical
Word 7 6 Y   spelling & punctuation only; 4 false corrections
WordPerfect 14 6 Y   spelling & punctuation only; 4 false corrections
Overall performances

  (click link to close)

      Here are cumulative results for each grammar checker on all the tests except "American and British English" (only
      a few checkers attempt to identify US-UK differences).  The "problems in sentences" test above is probably the best
      indicator of how well a given checker compares with others in its primary function, but the other grammar and style
      tests show comparative checking abilities across a range of typical grammar-checking duties.

      The percentages show what proportion of grammatical and stylistic problems each checker found and corrected
      (green numbers).  The percentages are always below 100: no computer program can find all the writing problems
      in the test documents.

      Note that the more items a checker flags, but either fails to correct (blue) or wrongly identifies as mistakes (red), the
      less helpful that checker will be to ESL and other writers who may have difficulty deciding between good advice and bad.
      We discuss this matter further in "Notes and Observations," below.     
  Test results totals for:                    

                GRAMMAR (366)     STYLE (379)

After the Deadline 51 14 21   14% 53 8 23   14%
AutoCrit 28 0 1   8% 32 0 3   8%
ClearEdits 6 3 9   <2% 65 10 7   17%; checks little besides wordiness
Correct English 105 30 50   29% 90 4 21   24%
Editor 286 9 16   78% 269 3 13   71%
Ginger 49 23 17   13% 1 3 28   does not check style
GrammarBase
61 19 24   17% 67 7 23   18%
GrammarCheck.net 55 13 22   15% 55 8 18   15%
Grammar Check Anywhere 47 23 61   13% 67 9 34   18%
Grammar Expert Plus 50 31 44   14% 66 6 21   17%
Grammarian Pro (Mac) 107 9 26   29% 62 0 2   16%
Grammarly 74 35 48   20% 40 2 44   11%
Grammar Slammer 55 17 22   15% 60 0 18   16%
PaperRater 33 15 15   9% 42 1 13   11%
Pro Writing Aid 56 17 11   15% 76 1 3   20%
SpellCheckPlus 82 31 38   22% 12 1 27   3%
StyleWriter 79 35 20   22% 123 17 17   32%
WhiteSmoke 119 18 45   33% 50 6 47   13%
  
Word 97 23 12   27% 30 2 18   8%
WordPerfect 104 45 28   28% 0 3 11   does not check style

Other Checking Programs
Notes and Observations
Disclaimer

Last revised 31 July 2014