Evaluation

Participants in any one or multiple task are welcome. Performance evaluation and system ranking will be based on single task. Each task will be evaluated and ranked separately. The winner will be ranked based on the identification accuracies. The system which performs best for individual tasks will be declared the winner. The test dataset is a closed dataset and will be made available once the participants submit their systems for evaluation.
Accuracy will be calculated as follows,

Accuracy = (CC/GT) X 100

Where, CC is the number of correctly identified samples; GT is the ground truth or total number of samples.
The participants have to submit their systems / executable either in Windows (XP or Win 7) executable format or Linux/Unix. The command lines of the system accepted for the competition are as follows,

1. Task 1 (Tri-Scripts identification with Indian scripts):
ClassifyEngHin<3rdScriptsName> Task1TestImages.txt Task1Result.txt

Where, is one of the acronym from {Ben, Ori, Guj, Pun, Kan, Tam, Tel, Arb, Mam}
Task1TestImages.txt is the input file having a list of paths to each JPG image to be tested:
Example:
D:\CVSI2017\ Task1TestImages \Test1.jpg
D:\CVSI2017\ Task1TestImages \Test2.jpg

Task1Result.txt is a single file which should have the following format and saves the identification result for all the samples in the test dataset:
<testSample name> | <Identified script>

Example:
D:\CVSI2017\Task1TestImages\Test1.jpg |Eng
D:\CVSI2017\Task1TestImages\Test2.jpg |Ben


<Identified script> should be any one of the three character string from the set {Eng, Hin, Ben, Ori, Guj, Pun, Kan, Tam, Tel, Arb, Mam, Chi, Tha, Jap, Kor}

2. Task 2 (Tri-Scripts identification with Oriental scripts)
ClassifyEng<2ndScripName><3rdScriptsName> Task2TestImages.txt Task2Result.txt
Where,<2ndScriptName> and <3rdScriptName> is one of the acronym from {Chi, Tha, Jap, Kor}

3. Task 3 (North Indian Scripts identification)
ClassifyNorthIndianScripts Task3TestImages.txt Task3Result.txt

4. Task 4 (South Indian Scripts identification)
ClassifySouthIndianScripts Task4TestImages.txt Task4Result.txt

5. Task 5 (South Asian Scripts identification)
ClassifyOrientalScripts Task5TestImages.txt Task5Result.txt

6. Task 6 (All Scripts identification)
ClassifyAllScripts Task6TestImages.txt Task6Result.txt

All the input text files and the result files have the same format as mentioned in point 1, above.