$ cat projects/crispr-quantum/results.md
CRISPR networks — results
The paper's results table in full, the three models side by side, and what the numbers do and do not support.
summary
The Classical NNW was the fastest and most stable across epochs, requiring 1.12 s CPU time (4.07 s Wall Time), achieving epoch 20 loss of 0.0059 and accuracy of 0.9352, but an R2 of -0.5858.
The Partially Quantum Hybrid model (H-NNW) started weaker and improved substantially from epoch 1 to epoch 20, reaching an accuracy of 0.9750, MSE (final) of 0.0188, MAE (final) of 0.0801 and an R2 of -0.0303, which the paper describes as the closest to positive among the quantum models. Its run took 20min 1s of CPU time and 20min 51s of wall time, an intermediate processing time that the paper says may be attributed to the additional complexity of the quantum processing layer and the simulation involved.
The Fully Quantum Intermediate model (HQNNW) also achieved 0.9750 accuracy and an R2 of -0.3112, but required 1 h 17 min 22 s CPU time and 1 h 18 min 59 s Wall Time without outperforming H-NNW.
table 2
| metric | Classical epoch 1 | Classical epoch 20 | H-NNW epoch 1 | H-NNW epoch 20 | HQNNW epoch 1 | HQNNW epoch 20 |
|---|---|---|---|---|---|---|
| Loss | 0.0070 | 0.0059 | 0.1805 | 0.0191 | 0.5564 | 0.0200 |
| Accuracy | 0.9182 | 0.9352 | 0.0386 | 0.9750 | 0.0750 | 0.9750 |
| MAE | 0.0517 | 0.0483 | 0.2736 | 0.0798 | 0.4784 | 0.0853 |
| MSE | 0.0070 | 0.0059 | 0.1805 | 0.0191 | 0.5564 | 0.0200 |
| MAPE | 161.5174 | 156.8017 | 319.6315 | 183.6465 | 993.0101 | 200.2417 |
| MSLE | 0.0041 | 0.0036 | 0.0796 | 0.0097 | 0.1039 | 0.0101 |
| Cosine Similarity | 0.9867 | 0.9886 | 0.5031 | 0.9641 | -0.2423 | 0.9631 |
| Log-cosh | 0.0034 | 0.0029 | 0.0811 | 0.0093 | 0.2055 | 0.0097 |
| MSE (final) | — | 0.0444 | — | 0.0188 | — | 0.0196 |
| MAE (final) | — | 0.1054 | — | 0.0801 | — | 0.0847 |
| R² | — | -0.5858 | — | -0.0303 | — | -0.3112 |
| CPU Time | — | 1.12 s | — | 20min 1s | — | 1h 17min 22s |
| Wall Time | — | 4.07 s | — | 20min 51s | — | 1h 18min 59s |
reading it
All three R² values are negative: -0.5858 for the classical network, -0.0303 for H-NNW and -0.3112 for HQNNW. An R² below zero means that predicting the mean every time would have scored better, so none of the three beats that trivial baseline on this measure.
The measures disagree on the winner. The classical network ends training with the lowest loss (0.0059) but has the highest final MSE (0.0444) and the most negative R². Both hybrids end with the higher accuracy (0.9750).
Every time in the table was measured on classical hardware. For H-NNW and HQNNW that includes simulating the quantum circuits, so those times measure the simulator, not a quantum device.
Not repeated from the paper: that the hybrid's R² is about 15 times the classical network's. Table 2 gives a ratio of about 19, and a ratio between negative R² values has no clear meaning. Two remarks in the paper's discussion are also left out because Table 2 does not bear them out: that the classical network had the better accuracy (its epoch-20 accuracy, 0.9352, is the lowest of the three; the lower loss, 0.0059, does hold), and that HQNNW showed no significant improvement by the last epoch (its loss falls from 0.5564 to 0.0200).
source
Marcus Navarro Gabrich, Henrique Cota de Freitas, Matheus Alcântara Souza. Análise de Redes Neurais para CRISPR: Uma Abordagem com Computação Quântica (Neural Network Analysis for CRISPR: A Quantum Computing Approach). Anais do XXV Simpósio em Sistemas Computacionais de Alto Desempenho (SSCAD 2024), pp. 13–24, Sociedade Brasileira de Computação. doi:10.5753/sscad.2024.244778