Visual Question Answering On Iconqa
Metrics
Reasoning (Alg.)
Reasoning (Com.)
Reasoning (Cou.)
Reasoning (Est.)
Reasoning (Fra.)
Reasoning (Geo.)
Reasoning (Mea.)
Reasoning (Pat.)
Reasoning (Pro.)
Reasoning (Sce.)
Reasoning (Sen.)
Reasoning (Spa.)
Reasoning (Tim.)
Sub-tasks (Blank)
Sub-tasks (Img.)
Sub-tasks (Txt.)
Results
Performance results of various models on this benchmark
Model Name | Reasoning (Alg.) | Reasoning (Com.) | Reasoning (Cou.) | Reasoning (Est.) | Reasoning (Fra.) | Reasoning (Geo.) | Reasoning (Mea.) | Reasoning (Pat.) | Reasoning (Pro.) | Reasoning (Sce.) | Reasoning (Sen.) | Reasoning (Spa.) | Reasoning (Tim.) | Sub-tasks (Blank) | Sub-tasks (Img.) | Sub-tasks (Txt.) | Paper Title | Repository |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DFAF | 50.27 | 81.69 | 70.68 | 99.02 | 77.60 | 81.80 | 98.83 | 56.60 | 85.70 | 67.01 | 84.11 | 51.42 | 67.72 | 78.28 | 77.72 | 72.17 | IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning | |
| Q-Only | 28.02 | 48.19 | 33.63 | 40.46 | 33.06 | 38.03 | 38.07 | 33.66 | 40.76 | 35.37 | 45.25 | 37.14 | 48.09 | 28.45 | 41.64 | 36.86 | IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning | |
| ViLBERT | 50.62 | 75.60 | 71.05 | 99.22 | 74.09 | 80.05 | 99.07 | 62.78 | 70.94 | 58.52 | 81.78 | 49.46 | 66.72 | 77.08 | 76.66 | 70.47 | IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning | |
| I-Only | 31.73 | 45.26 | 37.64 | 62.29 | 32.48 | 38.71 | 64.02 | 36.29 | 37.51 | 35.47 | 45.25 | 37.52 | 47.37 | 46.65 | 41.56 | 36.02 | IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning | |
| Random | 11.12 | 41.20 | 18.38 | 3.62 | 34.84 | 30.30 | 0.36 | 34.81 | 38.81 | 34.25 | 45.16 | 36.49 | 35.82 | 0.29 | 41.70 | 36.87 | IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning | |
| ViLT | 50.55 | 84.95 | 71.13 | 99.02 | 75.81 | 82.61 | 98.91 | 59.22 | 87.65 | 66.72 | 86.10 | 53.38 | 69.99 | 79.27 | 79.67 | 72.69 | IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning | |
| UNITER | 49.18 | 83.67 | 71.01 | 99.41 | 78.37 | 81.31 | 99.38 | 60.81 | 87.84 | 61.25 | 86.10 | 48.34 | 69.77 | 78.53 | 78.71 | 72.39 | IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning | |
| MCAN | 47.32 | 82.73 | 68.94 | 99.08 | 76.20 | 79.86 | 98.99 | 54.79 | 84.87 | 62.49 | 83.25 | 49.70 | 68.00 | 74.52 | 77.36 | 71.25 | IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning | |
| BAN | 47.46 | 82.12 | 67.56 | 97.06 | 73.77 | 79.99 | 96.50 | 55.67 | 82.45 | 66.92 | 82.12 | 53.20 | 66.50 | 75.54 | 76.33 | 70.82 | IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning | |
| Patch-TRM | 56.73 | 87.00 | 77.81 | 98.24 | 82.13 | 81.87 | 97.98 | 68.75 | 95.73 | 62.39 | 92.49 | 55.62 | 77.98 | 83.62 | 82.66 | 75.19 | IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning | |
| ViT | 51.10 | 82.12 | 70.84 | 98.95 | 77.41 | 82.60 | 98.76 | 58.46 | 86.07 | 68.80 | 84.72 | 54.64 | 68.66 | 78.92 | 79.15 | 72.34 | IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning | |
| Top-Down | 50.00 | 80.65 | 65.01 | 99.54 | 72.43 | 80.07 | 99.46 | 55.01 | 83.75 | 58.22 | 84.54 | 45.78 | 68.28 | 73.03 | 75.92 | 68.51 | IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning |
0 of 12 row(s) selected.