Topologically ordered lists of kanji for effective learning
30 seconds explanation for people who want to learn kanji:
It is best to learn kanji starting from simple characters and then learning complex ones as compositions of "parts", which are called "radicals" or "components". For example:
- 一 → 二 → 三
- 丨 → 凵 → 山 → 出
- 言 → 五 → 口 → 語
It is also smart to learn more common kanji first.
This project is based on those two ideas and provides properly ordered lists of kanji to make your learning process as fast, simple, and effective as possible.
Motivation for this project initially came from reading this article: The 5 Biggest Mistakes People Make When Learning Kanji.
First 100 kanji from lists/aozora.txt (formatted for convenience):
人一丨口日目儿見凵山
出十八木未丶来大亅了
子心土冂田思二丁彳行
寸寺時卜上丿刀分厶禾
私中彐尹事可亻何自乂
又皮彼亠方生月門間扌
手言女本乙气気干年三
耂者刂前勹勿豕冖宀家
今下白勺的云牛物立小
文矢知入乍作聿書学合
These lists can be found in lists
directory. They only differ in order of kanji. Each file contains a list of kanji, ordered as described in following sections. There are few options (see Used data for details):
aozora.(json|txt)
- ordered by kanji frequency in Japanese fiction and non-fiction books; I recommend this list if you're starting to learn kanjinews.(json|txt)
- ordered by kanji frequency in online newstwitter.(json|txt)
- ordered by kanji frequency in Twitter messageswikipedia.(json|txt)
- ordered by kanji frequency in Wikipedia articlesall.(json|txt)
- combined "average" version of all previous; this one is experimental, I don't recommend using itYou can use these lists to build an Anki deck or just as a guidance. If you're looking for "names" or meanings of kanji, you might want to check my kanji-keys project.
If you look at a kanji like 語, you can see it consists of at least three distinct parts: 言, 五, 口. Those are kanji by themselves too. The idea behind this project is to find the order of about 2000-2500 common kanji, in which no kanji appears before its' parts, so you only learn a new kanji when you already know its' components.
Topological sorting is done by using a modified version of Kahn (1962) algorithm with intermediate sorting step which deals with the second property above. This intermediate sorting uses the "weight" of each character: common kanji (lighter) tend appear before rare kanji (heavier). See source code for details.
Initial unsorted list contains only kanji which are present in KanjiVG project, so for each character there is a data of its' shape and stroke order.
Characters are split into components using CJK Decompositions Data project, along with "fixes" to simplify final lists and avoid characters which are not present in initial list.
Statistical data of kanji usage frequencies was collected by processing raw textual data from various sources. See kanji-frequency repository for details.
Kanji list covers about 95-99% of kanji found in various Japanese texts. Generally, the goal is provide something similar to Jōyō kanji, but based on actual data. Radicals are also included, but only those which are parts of some kanji in the list.
Kanji/radical must NOT appear in this list if it is:
( ^ω^)个
lists
directoryFiles in lists
directory are final lists.
*.txt
files contain lists as plain text, one character per line; those files can be interpreted as CSV/TSV files with a single column*.json
files contain lists as JSON arraysAll files are encoded in UTF-8, without byte order mark (BOM), and have unix-style line endings, LF
.
dependencies
directoryFiles in dependencies
directory are "flat" equivalents of CJK-decompositions (see below). "Dependency" here roughly means "a component of the visual decomposition" for kanji.
1-to-1.txt
has a format compatible with tsort command line utility; first character in each line is "target" kanji, second character is target's dependency or 0
1-to-1.json
contains a JSON array with the same data as in 1-to-1.txt
1-to-N.txt
is similar, but lists all "dependecies" at once1-to-N.json
contains a JSON object with the same data as in 1-to-N.txt
All files are encoded in UTF-8, without byte order mark (BOM), and have unix-style line endings, LF
.
data
directorykanji.json
- data for kanji included in final ordered lists, including radicals
kanjivg.txt
- list of kanji from KanjiVG
cjk-decomp-{VERSION}.txt
- data from CJK Decompositions Data, without any modificationscjk-decomp-override.txt
- data to override some CJK's decompositionskanji-frequency/*.json
- kanji frequency tablesAll files are encoded in UTF-8, without byte order mark (BOM). All files, except for cjk-decomp-{VERSION}.txt
, have unix-style line endings, LF
.
data/kanji.json
Contains table with data for kanji, including radicals. Columns are:
true
if it is a common kanjifalse
if it is primarily used as a radical/component and unlikely to be seen within top 3000 in kanji usage frequency tables. In this case character is only listed because it's useful for decomposition, not as a standalone kanjiResrictions:
kanjivg.txt
cjk-decomp-{VERSION}.txt
cjk-decomp-override.txt
data/kanjivg.txt
Simple list of characters which are present in KanjiVG project. Those are from the list of *.svg
files in KanjiVG's Github repository.
data/cjk-decomp-{VERSION}.txt
Data file from CJK Decompositions Data project, see description of its' format.
data/cjk-decomp-override.txt
Same format as cjk-decomp-{VERSION}.txt
, except:
#
allowedcjk-decomp-{VERSION}.txt
fix
, which just means "fix a record for the same character from original file"Special character 0
is used to distinguish invalid decompositions (which lead to characters with no graphical representation) from those which just can't be decomposed further into something meaningful. For example, 一:fix(0)
means that this kanji can't be further decomposed, since it's just a single stroke.
NOTE: Strictly speaking, records in this file are not always "visual decompositions" (but most of them are). Instead, it's just an attempt to provide meaningful recommendations of kanji learning order.
data/kanji-frequency/*.json
See kanji-frequency repository for details.
You must have Node.js and Git installed
git clone https://github.com/THIS/REPO.git
npm install
node build.js
+ commands and arguments described belowshow
- only display sorted list without writing into files
--per-line=NUM
- explicitly tell how many characters per line to display. 50
by default. Applicable only to (no arguments)--freq-table=TABLE_NAME
- use only one frequency table. Table names are file names from data/kanji-frequency
directory, without .json
extension, e.g. all
("combined" list), aozora
, etc. When omitted, all frequency tables are usedcoverage
- show tables coverage, i.e. which fraction of characters from each frequency table is included into kanji listsuggest-add
- suggest kanji to add in a list, based on coverage within kanji usage frequency tables
--num=NUM
- how many--mean-type=MEAN_TYPE
- same as previous, sort by given mean type: arithmetic
(most "extreme"), geometric
, harmonic
(default, most "conservative"). See Pythagorean means for detailssuggest-remove
- suggest kanji to remove from a list, reverse of suggest-add
--num=NUM
- see above--mean-type=MEAN_TYPE
- see abovesave
- update files with final listsThis is a multi-license project. Choose any license from this list: