Question regarding input sequence length standardization and cropping strategies in Figure 2a model comparisons

[skundei]

I would like to ask: In the comparison section of the paper, when comparing with other models, are the input sequence crops and lengths consistent across all models? Since the original papers for those models often use inconsistent cropping strategies and lengths, how was this issue addressed?

[bernardo-de-almeida]

Hi @skundei , can you be more precise in your question - what paper and comparisons do you mean? Please revise the Issue title as well to be more clear

[skundei]

Hi @bernardo-de-almeida,

Thank you for the quick response. I apologize for the ambiguity.

To clarify, I am referring to the paper "Nucleotide Transformer: building and evaluating robust foundation models for human genomics".

Specifically, my question concerns the benchmark comparison shown in Figure 2a, where the Nucleotide Transformer (NT) models are compared against supervised baselines (like BPNet) and other methods across the 18 genomic datasets (e.g., H2AFZ, Enhancers, Promoter tasks).

My detailed question is: Different models in the literature typically utilize vastly different input context lengths (e.g., DNABERT is often limited to 512bp, while others might use much longer contexts). When performing the comparisons for Figure 2a:

Consistency: Were the input sequence lengths and cropping strategies (e.g., center crop vs. sliding window) standardized across all models (including NT, BPNet, and others)?

Method: If they were standardized, did you re-train the baseline models on these specific cropped datasets to ensure fair comparison, or did you adapt the pre-trained versions of those models to the standardized input length?

I am asking to understand how you addressed the discrepancy in "native" input lengths between different architectures to ensure a fair benchmark.

I have updated the issue title as requested. Thanks!