In [1]:
cd ..
You have to run the whole pipeline as below at first.
In [2]:
sbt "run-main uima.cpe.CPERunner"
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384m; support was removed in 8.0
[info] Loading project definition from /Users/sakamoto/Documents/FelisCatusZero/project
[info] Set current project to FelisCatusZero (in build file:/Users/sakamoto/Documents/FelisCatusZero/)
[info] Running uima.cpe.CPERunner
[info] >> CPE Processing
[info] >> Question Analyzer Initializing
[info] >> Gzipped XMI Cas Consumer Initializing
[info] output CAS as Gzipped XMI into out/xmi/qa
[info] >> Information Retriever Initializing
[info] >> Gzipped XMI Cas Consumer Initializing
[info] output CAS as Gzipped XMI into out/xmi/ir
[info] >> Essay Generator Initializing
[info] >> Essay Writer Initializing
[info] >> Gzipped XMI Cas Consumer Initializing
[info] output CAS as Gzipped XMI into out/xmi/eg
[info] >> Essay Evaluator Initializing
[info] >> Essay Question Reader Initializing
[info] CPM Initialization Complete
[info] >> Essay Question Reader Processing
[info] >> Flow Controller Processing
[info] >> Question Analyzer Flow Controller Processing
[info] >> Question Analyzer Processing
[info] >> Question Analyzer Flow Controller Processing
[info] >> Gzipped XMI Cas Consumer Processing
[info] >> Question Analyzer Flow Controller Processing
[info] >> Flow Controller Processing
[info] >> Information Retriever Flow Controller Processing
[info] >> Information Retriever Processing
[info] #2(ジ ズ ヤ)
[info] #2(首 長 法)
[info] #2(ダ ラ イ ラ マ)
[info] #2(ナ ン ト の 王 令 廃 止)
[info] #2(ナ ン ト の 王 令 廃 止)
[info] #2(ミ ッ レ ト)
[info] #2(理 藩 院)
[info] #2(領 邦 教 会 制)
[info] #2(イ ン ド)
[info] #2(ア ッ バ ー ス 朝)
[info] #2(イ ブ ン シ ー ナ ー)
[info] #2(ア リ ス ト テ レ ス)
[info] #2(医 学)
[info] #2(代 数 学)
[info] #2(ト レ ド)
[info] #2(シ チ リ ア の 島)
[info] #2(シ チ リ ア が 島)
[info] #2(シ チ リ ア を 島)
[info] #2(シ チ リ ア に 島)
[info] #2(シ チ リ ア で 島)
[info] #2(シ チ リ ア 島)
[info] #2(シ チ リ ア)
[info] 60 つく ローマの平和 公共施設 字以内 支える 為る為さる 示す 繁栄 説明 都市生活
[info] #2(ス エ ズ の 運 河)
[info] #2(ス エ ズ が 運 河)
[info] #2(ス エ ズ を 運 河)
[info] #2(ス エ ズ に 運 河)
[info] #2(ス エ ズ で 運 河)
[info] #2(ス エ ズ 運 河)
[info] #2(ス エ ズ)
[info] #2(汽 船)
[info] #2(バ グ ダ ー ド 鉄 道)
[info] #2(モ ー ル ス 信 号)
[info] #2(マ ル コ 一 ニ)
[info] #2(義 和 団)
[info] #2(日 露 戦 争)
[info] #2(イ ラ ン の 立 憲 革 命)
[info] #2(イ ラ ン が 立 憲 革 命)
[info] #2(イ ラ ン を 立 憲 革 命)
[info] #2(イ ラ ン に 立 憲 革 命)
[info] #2(イ ラ ン で 立 憲 革 命)
[info] #2(イ ラ ン 立 憲 革 命)
[info] #2(イ ラ ン)
[info] #2(ガ ン デ ィ ー)
[info] #2(大 西 洋 憲 章)
[info] #2(日 本 国 憲 法)
[info] #2(台 湾)
[info] #2(金 日 成)
[info] #2(東 ド イ ツ)
[info] #2(E E C ( ヨ ー ロ ッ パ 経 済 共 同 体 ))
[info] #2(ヨ ー ロ ッ パ 経 済 共 同 体)
[info] #2(E E C)
[info] #2(ア ウ シ ュ ヴ ィ ッ ツ)
[info] #2(パ レ ス チ ナ の 難 民)
[info] #2(パ レ ス チ ナ が 難 民)
[info] #2(パ レ ス チ ナ を 難 民)
[info] #2(パ レ ス チ ナ に 難 民)
[info] #2(パ レ ス チ ナ で 難 民)
[info] #2(パ レ ス チ ナ 難 民)
[info] #2(パ レ ス チ ナ)
[info] >> Information Retriever Flow Controller Processing
[info] >> Gzipped XMI Cas Consumer Processing
[info] >> Information Retriever Flow Controller Processing
[info] >> Flow Controller Processing
[info] >> Essay Generator Flow Controller Processing
[info] >> Essay Generator Processing
[info] >> Essay Generator Flow Controller Processing
[info] >> Essay Writer Processing
[info] >> Essay Generator Flow Controller Processing
[info] >> Essay Evaluator Processing
[info] Evaluation Results
[info] 2017-01-25T15:00:11.826+09:00
[info] HOME
[info] Summary
[info]
[info] WRITER ROUGE AVERAGING # OF DATASETs # OF Qs MEAN MAX MEDIAN MIN VARIANCE STANDARD DEVIATION
[info] Baseline ROUGE-1 F1 Score by Content Word Micro 2 5 0.05470 0.08152 0.07025 0.00000 0.00085 0.02923
[info] Macro 2 5 0.05572 0.07670 0.07106 0.02492 0.00071 0.02354
[info] ROUGE-2 F1 Score by Content Word Micro 2 5 0.00241 0.00888 0.00000 0.00000 0.00001 0.00346
[info] Macro 2 5 0.00275 0.00602 0.00444 0.00000 0.00001 0.00296
[info]
[info]
[info] Particulars
[info]
[info] DATASET QUESTION LABEL HAS KEYWORDS? CHAR LIMIT WRITER ROUGE-1 F1 Score by Content Word ROUGE-2 F1 Score by Content Word
[info] A.xml K792W10_【１】 Yes 0 - 600 Baseline 0.07025 0.00000
[info] M792W10_【１】 Yes 0 - 510 Baseline 0.08152 0.00316
[info] M792W10_【２】_問(1)_(a) No 0 - 60 Baseline 0.00000 0.00000
[info] B.xml D792W10_【１】 Yes 0 - 510 Baseline 0.07187 0.00888
[info] F792W10_【１】 Yes 0 - 510 Baseline 0.04984 0.00000
[info]
[info]
[info] Evaluation Result History
[info]
[info] 2017-01-25T15:00:11.826+09:00
[info]
[info]
[info] >> Essay Generator Flow Controller Processing
[info] >> Gzipped XMI Cas Consumer Processing
[info] >> Essay Generator Flow Controller Processing
[info] >> Flow Controller Processing
[info] Completed 1 documents
[info] Total Time Elapsed: 681589565444 nano seconds
[info] Initialization Time: 709406159 nano seconds
[info] Processing Time: 680880159285 nano seconds
[info]
[info]
[info] ------------------ PERFORMANCE REPORT ------------------
[info]
[info] Component Name: Essay Question Collection Reader Descriptor
[info] Event Type: Process
[info] Duration: 215ms (0.03%)
[info] Result: success
[info] Component Name: aaeDescriptor
[info] Event Type: Analysis
[info] Duration: 680599ms (99.97%)
[info] Sub-events:
[info] Component Name: questionAnalyzerAAEDescriptor
[info] Event Type: Analysis
[info] Duration: 2160ms (0.32%)
[info] Sub-events:
[info] Component Name: questionAnalyzerAEDescriptor
[info] Event Type: Analysis
[info] Duration: 2058ms (0.3%)
[info]
[info] Component Name: Gzipped XMI Writer Cas Consumer
[info] Event Type: Analysis
[info] Duration: 100ms (0.01%)
[info]
[info] Component Name: Question Analyzer Flow Controller
[info] Event Type: Analysis
[info] Duration: 2ms (0%)
[info]
[info]
[info] Component Name: informationRetrievalAAEDescriptor
[info] Event Type: Analysis
[info] Duration: 423453ms (62.2%)
[info] Sub-events:
[info] Component Name: informationRetrievalAEDescriptor
[info] Event Type: Analysis
[info] Duration: 420095ms (61.7%)
[info]
[info] Component Name: Gzipped XMI Writer Cas Consumer
[info] Event Type: Analysis
[info] Duration: 3354ms (0.49%)
[info]
[info] Component Name: Information Retriever Flow Controller
[info] Event Type: Analysis
[info] Duration: 4ms (0%)
[info]
[info]
[info] Component Name: essayGeneratorAAEDescriptor
[info] Event Type: Analysis
[info] Duration: 254977ms (37.45%)
[info] Sub-events:
[info] Component Name: essayGeneratorAEDescriptor
[info] Event Type: Analysis
[info] Duration: 244080ms (35.85%)
[info]
[info] Component Name: Essay Writer Cas Consumer Descriptor
[info] Event Type: Analysis
[info] Duration: 71ms (0.01%)
[info]
[info] Component Name: Gzipped XMI Writer Cas Consumer
[info] Event Type: Analysis
[info] Duration: 3339ms (0.49%)
[info]
[info] Component Name: Evaluator Cas Consumer Descriptor
[info] Event Type: Analysis
[info] Duration: 7473ms (1.1%)
[info]
[info] Component Name: Essay Generator Flow Controller
[info] Event Type: Analysis
[info] Duration: 14ms (0%)
[info]
[info]
[info] Component Name: Flow Controller
[info] Event Type: Analysis
[info] Duration: 8ms (0%)
[info]
[info] Component Name: aaeDescriptor
[info] Event Type: End of Batch
[info] Duration: 0ms (0%)
[info]
[success] Total time: 683 s, completed Jan 25, 2017 3:11:34 PM
out/result/2017-01-25T15:00:11.826+09:00/evaluation_result.html is automatically opened by the default web browswer.
If the "HISTORY" link in the above page is clicked, out/result/index.html is shown as below.
If you want to run the pipeline from Essay Generator, do as below.
In [3]:
sbt "run-main uima.cpe.CPERunner -from eg"
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384m; support was removed in 8.0
[info] Loading project definition from /Users/sakamoto/Documents/FelisCatusZero/project
[info] Set current project to FelisCatusZero (in build file:/Users/sakamoto/Documents/FelisCatusZero/)
[info] Running uima.cpe.CPERunner -from eg
[info] >> CPE Processing
[info] >> Question Analyzer Initializing
[info] >> Gzipped XMI Cas Consumer Initializing
[info] output CAS as Gzipped XMI into out/xmi/qa
[info] >> Information Retriever Initializing
[info] >> Gzipped XMI Cas Consumer Initializing
[info] output CAS as Gzipped XMI into out/xmi/ir
[info] >> Essay Generator Initializing
[info] >> Essay Writer Initializing
[info] >> Gzipped XMI Cas Consumer Initializing
[info] output CAS as Gzipped XMI into out/xmi/eg
[info] >> Essay Evaluator Initializing
[info] >> Gzipped XMI Collection Reader Initializing
[info] input CAS from Gzipped XMI out/xmi/qa/0.xmi.gz
[info] >> Collection Reader Reconfiguration Started
[info] >> Gzipped XMI Collection Reader Initializing
[info] input CAS from Gzipped XMI out/xmi/ir/0.xmi.gz
[info] CPM Initialization Complete
[info] >> Gzipped XMI Collection Reader Processing
[info] >> Flow Controller Processing
[info] >> Essay Generator Flow Controller Processing
[info] >> Essay Generator Processing
[info] >> Essay Generator Flow Controller Processing
[info] >> Essay Writer Processing
[info] >> Essay Generator Flow Controller Processing
[info] >> Essay Evaluator Processing
[info] Evaluation Results
[info] 2017-01-25T15:12:53.784+09:00
[info] HOME
[info] Summary
[info]
[info] WRITER ROUGE AVERAGING # OF DATASETs # OF Qs MEAN MAX MEDIAN MIN VARIANCE STANDARD DEVIATION
[info] Baseline ROUGE-1 F1 Score by Content Word Micro 2 5 0.05470 0.08152 0.07025 0.00000 0.00085 0.02923
[info] Macro 2 5 0.05572 0.07670 0.07106 0.02492 0.00071 0.02354
[info] ROUGE-2 F1 Score by Content Word Micro 2 5 0.00241 0.00888 0.00000 0.00000 0.00001 0.00346
[info] Macro 2 5 0.00275 0.00602 0.00444 0.00000 0.00001 0.00296
[info]
[info]
[info] Particulars
[info]
[info] DATASET QUESTION LABEL HAS KEYWORDS? CHAR LIMIT WRITER ROUGE-1 F1 Score by Content Word ROUGE-2 F1 Score by Content Word
[info] A.xml K792W10_【１】 Yes 0 - 600 Baseline 0.07025 0.00000
[info] M792W10_【１】 Yes 0 - 510 Baseline 0.08152 0.00316
[info] M792W10_【２】_問(1)_(a) No 0 - 60 Baseline 0.00000 0.00000
[info] B.xml D792W10_【１】 Yes 0 - 510 Baseline 0.07187 0.00888
[info] F792W10_【１】 Yes 0 - 510 Baseline 0.04984 0.00000
[info]
[info]
[info] Evaluation Result History
[info]
[info] 2017-01-25T15:12:53.784+09:00
[info] 2017-01-25T15:00:11.826+09:00
[info]
[info] >> Essay Generator Flow Controller Processing
[info] >> Gzipped XMI Cas Consumer Processing
[info] >> Essay Generator Flow Controller Processing
[info] >> Flow Controller Processing
[info] Completed 1 documents
[info] Total Time Elapsed: 239531295507 nano seconds
[info] Initialization Time: 705371352 nano seconds
[info] Processing Time: 238825924155 nano seconds
[info]
[info]
[info] ------------------ PERFORMANCE REPORT ------------------
[info]
[info] Component Name: Gzipped XMI Collection Reader
[info] Event Type: Process
[info] Duration: 9464ms (3.96%)
[info] Result: success
[info] Component Name: aaeDescriptor
[info] Event Type: Analysis
[info] Duration: 229326ms (96.04%)
[info] Sub-events:
[info] Component Name: questionAnalyzerAAEDescriptor
[info] Event Type: Analysis
[info] Duration: 0ms (0%)
[info] Sub-events:
[info] Component Name: questionAnalyzerAEDescriptor
[info] Event Type: Analysis
[info] Duration: 0ms (0%)
[info]
[info] Component Name: Gzipped XMI Writer Cas Consumer
[info] Event Type: Analysis
[info] Duration: 0ms (0%)
[info]
[info] Component Name: Question Analyzer Flow Controller
[info] Event Type: Analysis
[info] Duration: 0ms (0%)
[info]
[info]
[info] Component Name: informationRetrievalAAEDescriptor
[info] Event Type: Analysis
[info] Duration: 0ms (0%)
[info] Sub-events:
[info] Component Name: informationRetrievalAEDescriptor
[info] Event Type: Analysis
[info] Duration: 0ms (0%)
[info]
[info] Component Name: Gzipped XMI Writer Cas Consumer
[info] Event Type: Analysis
[info] Duration: 0ms (0%)
[info]
[info] Component Name: Information Retriever Flow Controller
[info] Event Type: Analysis
[info] Duration: 0ms (0%)
[info]
[info]
[info] Component Name: essayGeneratorAAEDescriptor
[info] Event Type: Analysis
[info] Duration: 229294ms (96.02%)
[info] Sub-events:
[info] Component Name: essayGeneratorAEDescriptor
[info] Event Type: Analysis
[info] Duration: 215093ms (90.08%)
[info]
[info] Component Name: Essay Writer Cas Consumer Descriptor
[info] Event Type: Analysis
[info] Duration: 147ms (0.06%)
[info]
[info] Component Name: Gzipped XMI Writer Cas Consumer
[info] Event Type: Analysis
[info] Duration: 4160ms (1.74%)
[info]
[info] Component Name: Evaluator Cas Consumer Descriptor
[info] Event Type: Analysis
[info] Duration: 9890ms (4.14%)
[info]
[info] Component Name: Essay Generator Flow Controller
[info] Event Type: Analysis
[info] Duration: 4ms (0%)
[info]
[info]
[info] Component Name: Flow Controller
[info] Event Type: Analysis
[info] Duration: 32ms (0.01%)
[info]
[info] Component Name: aaeDescriptor
[info] Event Type: End of Batch
[info] Duration: 1ms (0%)
[info]
[success] Total time: 241 s, completed Jan 25, 2017 3:16:54 PM
out/result/2017-01-25T15:12:53.784+09:00/evaluation_result.html is automatically opened by the default web browswer.
If the HISTORY link in the above page is clicked, out/result/index.html is shown as below.
In [ ]:
Content source: ktr-skmt/FelisCatusZero
Similar notebooks: