Misc updates/fixes: labkey.domain.createAndLoad fix for strictFieldValidation, labkey.webdav.put description support, add labkey.query.import

[cnathe]

Rationale

Issue 43107: createAndLoad for Lists or Datasets using R returns an error
Issue 42950: Inserting file description using R API
Issue 42546: Improve performance of Rlabkey import for large number of rows

Related Pull Requests

• Misc updates/fixes: labkey.domain.createAndLoad fix for strictFieldValidation, labkey.webdav.put description support, add labkey.query.import #68
• Issue 43107: Add back strictFieldValidation property for study dataset and data class domain creation platform#2265
• RlabkeyTest - add test cases for labkey.domain.createAndLoad and labkey.query.import testAutomation#724

Changes

• Fix for labkey.domain.createAndLoad to set strictFieldValidation to false for study datasets and data classes
• Add param to labkey.webdav.put for setting the file description
• Add new labkey.query.import command which uses file import to bulk import rows to a query/table
• Update labkey.domain.createAndLoad to use the new labkey.query.import command for the bulk data import

[cnathe]

Here is the Rlabkey doc info for the new labkey.query.import command:

[labkey-adam]

Hey @cnathe , were you planning to update the labkey.setCurlOptions docs in this PR, per #67?

[cnathe]

Hey @cnathe , were you planning to update the labkey.setCurlOptions docs in this PR, per #67?

@labkey-adam Thanks for the reminder. I have it on my list. I was planning to get these changes in with this PR and then do a follow-on mini-PR which actually did the version bump and release notes. I had planned to include that docs update in that PR but can just as easily add it here.

[cnathe]

@labkey-klum What do you think of this change? The new labkey.query.import call is much faster then the labkey.insertRows or labkey.importRows calls. Do you recall any specific reason why the labkey.domain.createAndLoad would need to use the insertRows or do you think it is fine to change it to labkey.query.import?

[labkey-klum]

yes, agreed that changing this to use the new query.import function will improve the perf on this function.

[cnathe]

Test plan for changes in this PR (note that the fix for Issue 43107 includes a small server side change against 21.3):

Issue 43107: createAndLoad for Lists or Datasets using R returns an error

• verify that you can successfully use labkey.domain.createAndLoad for the following data types: datasets, list, sample type, data class

• verify that the domain is created and that the data rows are loaded as expected

• StudyDatasetVisit

labkey.domain.createAndLoad(
	baseUrl="http://localhost:8080/labkey", folderPath="Visit Based Study",
	name="Rlabkey dataset", df=dataset, domainKind="StudyDatasetVisit"
)

• list

 labkey.domain.createAndLoad(
	baseUrl="http://localhost:8080/labkey", folderPath="Visit Based Study",
	name="Rlabkey list", df=dataset, domainKind="IntList", options=list(keyName="Key")
)

• SampleSet

 labkey.domain.createAndLoad(
	baseUrl="http://localhost:8080/labkey", folderPath="Visit Based Study", 
	name="sampletype2", domainKind="SampleSet", df=df
)

• DataClass

labkey.domain.createAndLoad(
	baseUrl="http://localhost:8080/labkey", folderPath="Visit Based Study", 
	name="dataclass2", domainKind="DataClass", df=df
)

Issue 42950: Inserting file description using R API

• verify that the Rlabkey doc info for labkey.webdav.put has info on the new "description" param
• verify that you can pass a "description" string value to the labkey.webdav.put call and that it shows as expected on the LK server side

Issue 42546: Improve performance of Rlabkey import for large number of rows

• verify that the update Rlabkey package includes a new command for "labkey.query.import" and that the doc page for it looks as expected
• verify the error messages for missing required params to labkey.query.import command make sense
• verify usages of the labkey.query.import command without any options passed in
• verify usages of the labkey.query.import command with options passed in (see examples below for sample type merge)
• verify that the imported data looks as expected on the LK server side
• verify that error message from import are shown as expected in R console (i.e. trying to insert duplicate key or a missing required field value)

labkey.query.import(
	"http://localhost:8080/labkey",
	folderPath="/Visit Based Study", schemaName="samples", queryName="sampletype",
	toImport=df, options = list(insertOption = "MERGE", auditBehavior = "DETAILED")
)

[cnathe]

And here is a copy of the Rlabkey package built by TeamCity that can be installed locally for testing.
Rlabkey_2.6.0.tar.gz

[XingY]

And here is a copy of the Rlabkey package built by TeamCity that can be installed locally for testing.
Rlabkey_2.6.0.tar.gz

Can you please also attach the doc pdf for testing?