Clear page cache in some scripts

[rschu1ze]

Resolves #774

[puzpuzpuz]

It's still lukewarm since the Python process is kept around between query runs. The tag should be kept as is. Am I missing something?

[puzpuzpuz]

@rschu1ze thanks for handling the issue. Are you going to rerun the benchmarks as well? The results are likely to change after your modifications.

[rschu1ze]

I triggered the automation but all jobs fell victim to GitHub's outage today :-( Trying again now.

[puzpuzpuz]

Same as with polars, the lukewarm-cold-run tag should be kept. Worth checking other changed results too, e.g. it looks like duckdb-dataframe also keeps Python process around.

[puzpuzpuz]

I triggered the automation but all jobs fell victim to GitHub's outage today :-( Trying again now.

GH seems running normally now. Could you retry?

[rschu1ze]

Sure (however this PR is currently low priority for me, I work on it on the side, probably tomorrow or on the weekend).

[puzpuzpuz]

Sure (however this PR is currently low priority for me, I work on it on the side, probably tomorrow or on the weekend).

Sorry, I just wanted to check if this PR is still on your radar. No need to rush, I understand the circumstances.

[xe-nvdk]

Hey team, quick question. I thought that the tag was for the systems that do not clear OS cache and do not restart with every query. Am I wrong?

Forcing to all the systems restarting for every query is that in the roadmap yet?

Thank you

[nwoolmer]

Hey @xe-nvdk, my understanding is that lukewarm runs only clear OS page cache and do not require restarts. True cold runs require database restarts to ensure the internal database caches are also cleared.

Many results have not been migrated to 'true cold' runs yet, I think @rschu1ze wanted some help crowd-sourcing the refresh.

Citation:

2.a) True cold runs. Before each first run of each query, all operating system caches (page cache) and database caches (e.g. buffer pools) are cleared. 

Some databases provide commands to clear internal caches. For fairness towards databases which do not offer such statements, it is required to restart the database before the first run of each query. 

Databases which do not stick around as a background process between queries, e.g. [clickhouse-local](https://clickhouse.com/docs/operations/utilities/clickhouse-local), satisfy this requirement implicitly. 

It is still needed to clear the page cache before each first query to qualify as a true cold run.

[rschu1ze]

I somehow pushed the wrong branch to the wrong location, GitHub closed this PR and won't let me open it again.

[puzpuzpuz]

I somehow pushed the wrong branch to the wrong location, GitHub closed this PR and won't let me open it again.

I guess you could create a new branch and open a new PR.

[rschu1ze]

Replacement PR: #792

Re cold runs: Yes, the migration to "true" cold runs has not started yet, and we better crowd-source it. Let me create an umbrella ticket --> #793