Add ivscc_apfrequency operation

[MichaelHuth]

close #2581

[MichaelHuth]

@timjarsky This is a first version to play around. There are still a few things I have to add, that I discuss farther below.

ivscc_apfrequency([xaxisOffset, yaxisOffset, xAxisPercentage, yAxisPercentage]) where [] denotes that the parameters are optional.

xaxisOffset and yaxisOffset are strings that can be min, max and none.

General Plotting Behavior

The operation itself returns internally a full plotting specification that is inserted by the formula plotter at the location where the operation appears in the notebook code.

The operation creates only traces that are separated by with. This means preceding and succeeding formulas that are also separated with with go to the same subwindow.

xAxisPercentage and yAxisPercentage are treated as properties for the plot by the plotter. Specifically plot means in this context the subwindow where the traces go. Plot properties from the last formula in the with chain are applied. (i.e. there is currently no gathering of plot properties).

Thus, the plotter applies the 10% for x and y-axis when used like this, where the formula setting the plot properties is last in the chain:

1
with
ivscc_apfrequency(min, min, 10, 10)

but not for this:

ivscc_apfrequency(min, min, 10, 10)
with
1

because 1 does not include any plot properties.

Operation Arguments

Currently:

ivscc_apfrequency([xaxisOffset, yaxisOffset, xAxisPercentage, yAxisPercentage])

with xaxisOffset and yaxisOffset as min, max, none and xAxisPercentage, yAxisPercentage a number between 0 and 100.

The (later) final arguments should also expose arguments from apfrequency, so it will change to ivscc_apfrequency([xaxisOffset, yaxisOffset, xAxisPercentage, yAxisPercentage, method, level, resultType, normalize, xAxisType).

The default for xaxisOffset and yaxisOffset is min.

On the basis of the experiment avgMethodTesting2.pxp the generated code is:

sel = select(selsweeps(), selstimset("*rheo*", "*supra*"), selvis(all))
selexpAD0 = select(selexp("C57BL6J-734969.15.10A.01.nwb"), $sel, selchannels(AD0), selrange(E1))
selexpDA0 = select(selexp("C57BL6J-734969.15.10A.01.nwb"), $sel, selchannels(DA0), selrange(E1))
freq0 = apfrequency(data($selexpAD0))
current0 = max(data($selexpDA0))
currentNorm0 = $current0 - extract($current0, 0) #*1
selexpAD1 = select(selexp("C57BL6J-734969.15.10B.01.nwb"), $sel, selchannels(AD0), selrange(E1))
selexpDA1 = select(selexp("C57BL6J-734969.15.10B.01.nwb"), $sel, selchannels(DA0), selrange(E1))
freq1 = apfrequency(data($selexpAD1))
current1 = max(data($selexpDA1))
currentNorm1 = $current1 - extract($current1, 0) #*2
ivsccavg = avg([$freq0,$freq1], group)
ivscccurrentavg = avg([$currentNorm0,$currentNorm1], group)

$freq0 - extract($freq0, 0) #*3
vs
$currentNorm0
with
$freq1 - extract($freq1, 0) #*4
vs
$currentNorm1
with
$ivsccavg - extract($ivsccavg, 0) #*5
vs
$ivscccurrentavg - extract($ivscccurrentavg, 0) #*6

I added a #* to the formulas that change depending on the min, max, none setting.

#*1, #*2 and #*6 depend on the xaxisOffset argument with the following logic:

min: currentNormX = $currentX - extract($currentX, 0)
max: currentNormX = $currentX - max(flatten($currentX))
none: currentNormX = $currentX

for #*6 it is:
min: $ivscccurrentavg - extract($ivscccurrentavg, 0)
max: $ivscccurrentavg - max(flatten($ivscccurrentavg))
none: $ivscccurrentavg

#*3, #*4 and #*5 depend on the yaxisOffset argument with the following logic:

min: $freqX - extract($freqX, 0)
max: $freqX - max(flatten($freqX))
none: $freqX

for #*5 it is:
min: $ivsccavg - extract($ivsccavg, 0)
max: $ivsccavg - max(flatten($ivsccavg))
none: $ivsccavg

I need to add a flatten operation because the result in e.g. $freqX are single values in 7 datasets (for avgMethodTesting2.pxp). Thus, our max() operation would determine the max in each dataset individually, but that is not what is wanted. The flatten operation should change n datasets with a single data point to an array with n elements.

Therefore, the max argument is currently not implemented yet until I have the flatten operation implemented.

An additional task from the issue is to add a variable that contains the names of the experiments. I can create this variable in the operation and add it to the variable storage of the formula notebook. It would be available then after the operation ran.
For now I thought of $ivscc_apfrequency_explist as name and it would be a string array.

[timjarsky]

@MichaelHuth attempting to use ivscc_apfrequency() on the linked data , resulted in the following assertions:

  !!! Assertion FAILED !!!
  Message: "Input and output must have the same size."
  Please provide the following information if you contact the MIES developers:
  ################################
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Stacktrace:
  SF_button_sweepFormula_display(...)#L2158 [MIES_SweepFormula.ipf]
SF_FormulaPlotter(...)#L1401 [MIES_SweepFormula.ipf]
SF_GatherFormulaResults(...)#L255 [MIES_SweepFormula.ipf]
SFE_ExecuteFormula(...)#L47 [MIES_SweepFormula_Executor.ipf]
SFE_FormulaExecutor(...)#L520 [MIES_SweepFormula_Executor.ipf]
SFO_OperationIVSCCApFrequency(...)#L2570 [MIES_SweepFormula_Operations.ipf]
SFE_ExecuteVariableAssignments(...)#L89 [MIES_SweepFormula_Executor.ipf]
SFE_FormulaExecutor(...)#L348 [MIES_SweepFormula_Executor.ipf]
SFO_OperationAvg(...)#L519 [MIES_SweepFormula_Operations.ipf]
SFH_TransferFormulaDataWaveNoteAndMeta(...)#L974 [MIES_SweepFormula_Helpers.ipf]
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Time: 2025-12-17T12:58:37-08:00
  Locked device: [- none -]
  Current sweep: [- none -]
  DAQ: [- none -]
  Testpulse: [- none -]
  Acquisition state: [- none -]
  Experiment: Untitled ()
  Igor Pro version: 10.0.12.29764 (29764)
  MIES version:
  Release_2.9_20250502-557-gbd0beedf3-dirty
Date and time of last commit: 2025-12-17T19:14:41+01:00
Submodule status: 
-e812ca28993a2414192e48c572085db5f17a13d7 Packages/IPNWB
-5270cb8911ff888ae74f6d3ee2b7b0d49235f98b Packages/doc/doxygen-filter-ipf
-995e48203398ad085f372c0ec6b9f7cbb2bb0362 Packages/igortest
  ################################
  MIES BUG AD_GetSquarePulseFailMsg(...)#L486: Missing DAScale stepsize LBN entry
  !!! Assertion FAILED !!!
  Message: "Input and output must have the same size."
  Please provide the following information if you contact the MIES developers:
  ################################
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Stacktrace:
  AD_CheckProc_Toggle(...)#L1423 [MIES_AnalysisFunctions_Dashboard.ipf]
AdaptDependentControls(...)#L2300 [MIES_GuiUtilities.ipf]
PGC_SetAndActivateControl(...)#L294 [MIES_ProgrammaticGUIControl.ipf]
BSP_CheckProc_OverlaySweeps(...)#L1319 [MIES_BrowserSettingsPanel.ipf]
OVS_UpdatePanel(...)#L271 [MIES_OverlaySweeps.ipf]
OVS_EndIncrementalUpdate(...)#L851 [MIES_OverlaySweeps.ipf]
UpdateSweepPlot(...)#L669 [MIES_MiesUtilities_GUI.ipf]
SB_UpdateSweepPlot(...)#L333 [MIES_AnalysisBrowser_SweepBrowser.ipf]
PostPlotTransformations(...)#L781 [MIES_Browser_Plotter.ipf]
SF_Update(...)#L2107 [MIES_SweepFormula.ipf]
PGC_SetAndActivateControl(...)#L220 [MIES_ProgrammaticGUIControl.ipf]
SF_button_sweepFormula_display(...)#L2158 [MIES_SweepFormula.ipf]
SF_FormulaPlotter(...)#L1401 [MIES_SweepFormula.ipf]
SF_GatherFormulaResults(...)#L255 [MIES_SweepFormula.ipf]
SFE_ExecuteFormula(...)#L47 [MIES_SweepFormula_Executor.ipf]
SFE_FormulaExecutor(...)#L520 [MIES_SweepFormula_Executor.ipf]
SFO_OperationIVSCCApFrequency(...)#L2570 [MIES_SweepFormula_Operations.ipf]
SFE_ExecuteVariableAssignments(...)#L89 [MIES_SweepFormula_Executor.ipf]
SFE_FormulaExecutor(...)#L348 [MIES_SweepFormula_Executor.ipf]
SFO_OperationAvg(...)#L519 [MIES_SweepFormula_Operations.ipf]
SFH_TransferFormulaDataWaveNoteAndMeta(...)#L974 [MIES_SweepFormula_Helpers.ipf]
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Time: 2025-12-17T12:59:11-08:00
  Locked device: [- none -]
  Current sweep: [- none -]
  DAQ: [- none -]
  Testpulse: [- none -]
  Acquisition state: [- none -]
  Experiment: Untitled ()
  Igor Pro version: 10.0.12.29764 (29764)
  MIES version:
  Release_2.9_20250502-557-gbd0beedf3-dirty
Date and time of last commit: 2025-12-17T19:14:41+01:00
Submodule status: 
-e812ca28993a2414192e48c572085db5f17a13d7 Packages/IPNWB
-5270cb8911ff888ae74f6d3ee2b7b0d49235f98b Packages/doc/doxygen-filter-ipf
-995e48203398ad085f372c0ec6b9f7cbb2bb0362 Packages/igortest
  ################################
  ** a wave read gave error: Index out of range for wave "_free_".
  !!! Assertion FAILED !!!
  Message: "Input and output must have the same size."
  Please provide the following information if you contact the MIES developers:
  ################################
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Stacktrace:
  AD_CheckProc_Toggle(...)#L1423 [MIES_AnalysisFunctions_Dashboard.ipf]
AdaptDependentControls(...)#L2285 [MIES_GuiUtilities.ipf]
PGC_SetAndActivateControl(...)#L294 [MIES_ProgrammaticGUIControl.ipf]
BSP_CheckProc_OverlaySweeps(...)#L1319 [MIES_BrowserSettingsPanel.ipf]
OVS_UpdatePanel(...)#L271 [MIES_OverlaySweeps.ipf]
OVS_EndIncrementalUpdate(...)#L851 [MIES_OverlaySweeps.ipf]
UpdateSweepPlot(...)#L669 [MIES_MiesUtilities_GUI.ipf]
SB_UpdateSweepPlot(...)#L333 [MIES_AnalysisBrowser_SweepBrowser.ipf]
PostPlotTransformations(...)#L781 [MIES_Browser_Plotter.ipf]
SF_Update(...)#L2107 [MIES_SweepFormula.ipf]
PGC_SetAndActivateControl(...)#L220 [MIES_ProgrammaticGUIControl.ipf]
SF_button_sweepFormula_display(...)#L2158 [MIES_SweepFormula.ipf]
SF_FormulaPlotter(...)#L1401 [MIES_SweepFormula.ipf]
SF_GatherFormulaResults(...)#L255 [MIES_SweepFormula.ipf]
SFE_ExecuteFormula(...)#L47 [MIES_SweepFormula_Executor.ipf]
SFE_FormulaExecutor(...)#L520 [MIES_SweepFormula_Executor.ipf]
SFO_OperationIVSCCApFrequency(...)#L2570 [MIES_SweepFormula_Operations.ipf]
SFE_ExecuteVariableAssignments(...)#L89 [MIES_SweepFormula_Executor.ipf]
SFE_FormulaExecutor(...)#L348 [MIES_SweepFormula_Executor.ipf]
SFO_OperationAvg(...)#L519 [MIES_SweepFormula_Operations.ipf]
SFH_TransferFormulaDataWaveNoteAndMeta(...)#L974 [MIES_SweepFormula_Helpers.ipf]
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Time: 2025-12-17T12:59:22-08:00
  Locked device: [- none -]
  Current sweep: [- none -]
  DAQ: [- none -]
  Testpulse: [- none -]
  Acquisition state: [- none -]
  Experiment: Untitled ()
  Igor Pro version: 10.0.12.29764 (29764)
  MIES version:
  Release_2.9_20250502-557-gbd0beedf3-dirty
Date and time of last commit: 2025-12-17T19:14:41+01:00
Submodule status: 
-e812ca28993a2414192e48c572085db5f17a13d7 Packages/IPNWB
-5270cb8911ff888ae74f6d3ee2b7b0d49235f98b Packages/doc/doxygen-filter-ipf
-995e48203398ad085f372c0ec6b9f7cbb2bb0362 Packages/igortest
  ################################
  ** a wave read gave error: Index out of range for wave "_free_".
  MIES BUG AD_GetSquarePulseFailMsg(...)#L486: Missing DAScale stepsize LBN entry
  !!! Assertion FAILED !!!
  Message: "Input and output must have the same size."
  Please provide the following information if you contact the MIES developers:
  ################################
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Stacktrace:
  SF_button_sweepFormula_display(...)#L2158 [MIES_SweepFormula.ipf]
SF_FormulaPlotter(...)#L1401 [MIES_SweepFormula.ipf]
SF_GatherFormulaResults(...)#L255 [MIES_SweepFormula.ipf]
SFE_ExecuteFormula(...)#L47 [MIES_SweepFormula_Executor.ipf]
SFE_FormulaExecutor(...)#L520 [MIES_SweepFormula_Executor.ipf]
SFO_OperationIVSCCApFrequency(...)#L2570 [MIES_SweepFormula_Operations.ipf]
SFE_ExecuteVariableAssignments(...)#L89 [MIES_SweepFormula_Executor.ipf]
SFE_FormulaExecutor(...)#L348 [MIES_SweepFormula_Executor.ipf]
SFO_OperationAvg(...)#L519 [MIES_SweepFormula_Operations.ipf]
SFH_TransferFormulaDataWaveNoteAndMeta(...)#L974 [MIES_SweepFormula_Helpers.ipf]
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Time: 2025-12-17T13:04:21-08:00
  Locked device: [- none -]
  Current sweep: [- none -]
  DAQ: [- none -]
  Testpulse: [- none -]
  Acquisition state: [- none -]
  Experiment: Untitled ()
  Igor Pro version: 10.0.12.29764 (29764)
  MIES version:
  Release_2.9_20250502-557-gbd0beedf3-dirty
Date and time of last commit: 2025-12-17T19:14:41+01:00
Submodule status: 
-e812ca28993a2414192e48c572085db5f17a13d7 Packages/IPNWB
-5270cb8911ff888ae74f6d3ee2b7b0d49235f98b Packages/doc/doxygen-filter-ipf
-995e48203398ad085f372c0ec6b9f7cbb2bb0362 Packages/igortest
  ################################
  MIES BUG_TS: Encountered pending RTE: 1321, a wave read;Index out of range for wave "_free_".
  !!! Assertion FAILED !!!
  Message: "Variable not found"
  Please provide the following information if you contact the MIES developers:
  ################################
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Stacktrace:
  SF_button_sweepFormula_display(...)#L2171 [MIES_SweepFormula.ipf]
SF_MarkErrorLocationInNotebook(...)#L2760 [MIES_SweepFormula.ipf]
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Time: 2025-12-17T13:04:26-08:00
  Locked device: [- none -]
  Current sweep: [- none -]
  DAQ: [- none -]
  Testpulse: [- none -]
  Acquisition state: [- none -]
  Experiment: Untitled ()
  Igor Pro version: 10.0.12.29764 (29764)
  MIES version:
  Release_2.9_20250502-557-gbd0beedf3-dirty
Date and time of last commit: 2025-12-17T19:14:41+01:00
Submodule status: 
-e812ca28993a2414192e48c572085db5f17a13d7 Packages/IPNWB
-5270cb8911ff888ae74f6d3ee2b7b0d49235f98b Packages/doc/doxygen-filter-ipf
-995e48203398ad085f372c0ec6b9f7cbb2bb0362 Packages/igortest
  ################################
  ** FindDimLabel gave error: expected wave name
  ** a wave read gave error: Index out of range for wave "_free_".

[MichaelHuth]

I had to fill a request form from Google for the data.

[Copilot]

Pull request overview

This PR adds a new ivscc_apfrequency operation to the sweep formula system, along with supporting infrastructure changes including a flatten operation and refactored plotting code.

Key Changes

• Adds ivscc_apfrequency operation for analyzing action potential frequency in IV sweep current clamp experiments
• Introduces flatten helper operation to convert datasets of single values into 1D arrays
• Refactors plotting infrastructure by introducing SF_PlotterGraphStruct to encapsulate plotting state
• Adds axis offset and percentage control capabilities to the plotting metadata system

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 17 comments.

Show a summary per file

File	Description
Packages/MIES/MIES_Constants.ipf	Adds new metadata constants for plot customization and operation names (contains merge conflict)
Packages/MIES/MIES_SweepFormula_Operations.ipf	Implements flatten and ivscc_apfrequency operations
Packages/MIES/MIES_SweepFormula_Executor.ipf	Registers new operations in the executor switch statement
Packages/MIES/MIES_SweepFormula.ipf	Major refactoring of plotting code to use structure-based state management
Packages/MIES/MIES_SweepFormula_Helpers.ipf	Adds helper functions for internal formula execution and variable storage
Packages/MIES/MIES_WaveDataFolderGetters.ipf	Expands plot metadata wave to include axis offset/percentage properties

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[MichaelHuth]

@timjarsky I fixed the assertion that came up. The reason was that with the three experiments the selections result in 20 sweeps from the first experiment, 24 from the second and 16 from the third. The avg operation was transferring the wave notes and meta data always from the first group to the averaged result. And this failed because the averaged result always has the highest number of datasets (24 in this case) and I can not transfer from 20 datasets to 24 datasets as for the last 4 there would be not wave notes and meta data to transfer.
I changed the behavior of the avg operation in group mode now that it transfers the wave notes and meta data from the group with the highest number of datasets. For this case this means it is transferred from the second group.

For this data this also means that for the first 16 sweeps in the groups the average is over 3 sweeps, 16 - 19 over 2 sweeps and 20 - 23 over 1 sweep.

In the most recent version of this PR also the max argument works now as I added the flatten operation.

And the $ivscc_apfrequency_explist variable is created.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 14 comments.

Comments suppressed due to low confidence (1)

Packages/MIES/MIES_SweepFormula.ipf:1630

• Missing return statement: The function SF_AddPlotTraceStyle declares a return value of type SF_PlotterGraphStruct but has no explicit return statement. This will cause a compilation error or undefined behavior. Add 'return [pg]' at the end of the function.

static Function [STRUCT SF_PlotterGraphStruct pg] SF_AddPlotTraceStyle(variable formulasAreDifferent)

	variable i, j, numTraces, markerCode, lineCode, isCategoryAxis, tagCounter, lineStyle, overrideMarker, traceToFront
	string trace, info, tagText, name, wvName

	for(i = 0; i < pg.formulaCounter; i += 1)
		WAVE/WAVE plotFormData  = pg.collPlotFormData[i]
		WAVE/T    tracesInGraph = plotFormData[0]
		WAVE/WAVE dataInGraph   = plotFormData[1]
		numTraces  = DimSize(tracesInGraph, ROWS)
		markerCode = formulasAreDifferent ? i : 0
		markerCode = SFH_GetPlotMarkerCodeSelection(markerCode)
		lineCode   = formulasAreDifferent ? i : 0
		lineCode   = SFH_GetPlotLineCodeSelection(lineCode)
		for(j = 0; j < numTraces; j += 1)

			WAVE/Z wvX = dataInGraph[j][%WAVEX]
			WAVE   wvY = dataInGraph[j][%WAVEY]
			trace = tracesInGraph[j]

			info           = AxisInfo(pg.win, "left")
			isCategoryAxis = (NumberByKey("ISCAT", info) == 1)

			if(isCategoryAxis)
				WAVE traceColorHolder = wvX
			else
				WAVE traceColorHolder = wvY
			endif

			WAVE/Z traceColor = JWN_GetNumericWaveFromWaveNote(traceColorHolder, SF_META_TRACECOLOR)
			if(WaveExists(traceColor))
				switch(DimSize(traceColor, ROWS))
					case 3:
						ModifyGraph/W=$pg.win rgb($trace)=(traceColor[0], traceColor[1], traceColor[2])
						break
					case 4:
						ModifyGraph/W=$pg.win rgb($trace)=(traceColor[0], traceColor[1], traceColor[2], traceColor[3])
						break
					default:
						FATAL_ERROR("Invalid size of trace color wave")
				endswitch
			endif

			tagText = JWN_GetStringFromWaveNote(wvY, SF_META_TAG_TEXT)
			if(!IsEmpty(tagText))
				name = "tag" + num2str(tagCounter++)
				Tag/C/N=$name/W=$pg.win/F=0/L=0/X=0.00/Y=0.00 $trace, 0, tagText
			endif

			ModifyGraph/W=$pg.win mode($trace)=SF_DeriveTraceDisplayMode(wvX, wvY)

			lineStyle = JWN_GetNumberFromWaveNote(wvY, SF_META_LINESTYLE)
			if(IsValidTraceLineStyle(lineStyle))
				ModifyGraph/W=$pg.win lStyle($trace)=lineStyle
			elseif(formulasAreDifferent)
				ModifyGraph/W=$pg.win lStyle($trace)=lineCode
			endif

			WAVE/Z customMarkerAsFree = JWN_GetNumericWaveFromWaveNote(wvY, SF_META_MOD_MARKER)
			if(WaveExists(customMarkerAsFree))
				DFREF dfrWork = SFH_GetWorkingDF(pg.graph)
				wvName = "customMarker_" + NameOfWave(wvY)
				WAVE customMarker = MoveFreeWaveToPermanent(customMarkerAsFree, dfrWork, wvName)
				ASSERT(DimSize(wvY, ROWS) == DimSize(customMarker, ROWS), "Marker size mismatch")
				ModifyGraph/W=$pg.win zmrkNum($trace)={customMarker}
			else
				overrideMarker = JWN_GetNumberFromWaveNote(wvY, SF_META_MOD_MARKER)

				if(!IsNaN(overrideMarker))
					markerCode = overrideMarker
				endif

				ModifyGraph/W=$pg.win marker($trace)=markerCode
			endif

			traceToFront = JWN_GetNumberFromWaveNote(wvY, SF_META_TRACETOFRONT)
			traceToFront = IsNaN(traceToFront) ? 0 : !!traceToFront
			if(traceToFront)
				ReorderTraces/W=$pg.win _front_, {$trace}
			endif

		endfor
	endfor
End

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[timjarsky]

@MichaelHuth

Thanks for handling the metadata management for mismatched sweep numbers across experiments.

A few points for discussion:

1. The marker coloring makes it hard for me to evaluate the output (see plot image below). Can we use a single color for each experiment? Can the data be a single 1D wave instead of multiple x-y waves (this will enable fitting)?

2. There are many zero-frequency measurements spread across the x-axis.

3. The average seems more variable than the input data, perhaps because there are too many data points?

4. I'm not sure about the utility of the negative current values with "min" and "max" (second image). The two axes options that I'm sure are needed are none (where each FI curve starts at zero current and zero frequency) and the absolute current values.

ivscc_apfrequency(none, none)

ivscc_apfrequency()

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[MichaelHuth]

@timjarsky

1. I changed that. There are now experiment + 1 traces with markers. The +1 trace is the average in the average color. The experiment traces use the common table for trace colors, with one color per experiment. I also added code to show the experiment name in the legend for these.
2. I still have to look into this, why a lot of the results return a zero. Currently the standard setting of apfrequency for the internal apfrequency call are used.
3. Yes, it looks a bit like that. I did not see that in my local testing with the three experiments though. I guess in the graphs you pasted there is for some of the average points only a single sweep for an index in the groups.
   e.g. first exp has 10 and second exp has 11 sweeps selected, then sweep data for index 10 is only present from the second experiment -> only one sweep goes into the average and then the trace point from the experiment and the trace point from the average are equal. The average is shown in front and covers the point from the experiment. You may check for the average points on the very top if there is a data point from an experiment underneath.
4. I think it is the other way around, with none no offset is applied. min moves the trace to zero and max moves the trace such that the former maximum point is at zero.
   What I think might be unexpected with the initial formula construct is that there is an offset used per trace. Thus, when changing none -> min then each trace start is moved to zero separately and the visual relation between the traces shifts. (same applies for max)
   This per-trace behavior is the same for the xaxisOffset and yaxisOffset.

[MichaelHuth]

@timjarsky
Regarding 2.: The default level for apfrequency is 0. This results in zero peaks found for sweeps where E1 does not cross the zero line and for this apfrequency returns 0 as result with the default arguments.

I have added support for the apfrequency argument block after the first four argument for ivscc_apfrequency. The arguments are now:
ivscc_apfrequency([xaxisOffset, yaxisOffset, xAxisPercentage, yAxisPercentage, method, level, resultType, normalize, xAxisType])

The last four arguments are "forwarded" to apfrequency.

[timjarsky]

@MichaelHuth, Are failing sweeps included or filtered out? If included, please update to use only passing sweeps.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 9 comments.

Comments suppressed due to low confidence (1)

Packages/MIES/MIES_SweepFormula.ipf:1658

• The function signature declares it returns a structure [STRUCT SF_PlotterGraphStruct pg], but there is no return statement at the end of the function (line 1658). While the structure is passed by value in Igor Pro and modifications inside the function don't affect the caller unless explicitly returned, the function signature promises to return the structure but doesn't. This could lead to the caller receiving an uninitialized or default-valued structure instead of the modified one.

static Function [STRUCT SF_PlotterGraphStruct pg] SF_AddPlotTraceStyle(variable formulasAreDifferent)

	variable i, j, numTraces, markerCode, lineCode, isCategoryAxis, tagCounter, lineStyle, overrideMarker, traceToFront
	string trace, info, tagText, name, wvName

	for(i = 0; i < pg.formulaCounter; i += 1)
		WAVE/WAVE plotFormData  = pg.collPlotFormData[i]
		WAVE/T    tracesInGraph = plotFormData[0]
		WAVE/WAVE dataInGraph   = plotFormData[1]
		numTraces  = DimSize(tracesInGraph, ROWS)
		markerCode = formulasAreDifferent ? i : 0
		markerCode = SFH_GetPlotMarkerCodeSelection(markerCode)
		lineCode   = formulasAreDifferent ? i : 0
		lineCode   = SFH_GetPlotLineCodeSelection(lineCode)
		for(j = 0; j < numTraces; j += 1)

			WAVE/Z wvX = dataInGraph[j][%WAVEX]
			WAVE   wvY = dataInGraph[j][%WAVEY]
			trace = tracesInGraph[j]

			info           = AxisInfo(pg.win, "left")
			isCategoryAxis = (NumberByKey("ISCAT", info) == 1)

			if(isCategoryAxis)
				WAVE traceColorHolder = wvX
			else
				WAVE traceColorHolder = wvY
			endif

			WAVE/Z traceColor = JWN_GetNumericWaveFromWaveNote(traceColorHolder, SF_META_TRACECOLOR)
			if(WaveExists(traceColor))
				switch(DimSize(traceColor, ROWS))
					case 3:
						ModifyGraph/W=$pg.win rgb($trace)=(traceColor[0], traceColor[1], traceColor[2])
						break
					case 4:
						ModifyGraph/W=$pg.win rgb($trace)=(traceColor[0], traceColor[1], traceColor[2], traceColor[3])
						break
					default:
						FATAL_ERROR("Invalid size of trace color wave")
				endswitch
			endif

			tagText = JWN_GetStringFromWaveNote(wvY, SF_META_TAG_TEXT)
			if(!IsEmpty(tagText))
				name = "tag" + num2str(tagCounter++)
				Tag/C/N=$name/W=$pg.win/F=0/L=0/X=0.00/Y=0.00 $trace, 0, tagText
			endif

			ModifyGraph/W=$pg.win mode($trace)=SF_DeriveTraceDisplayMode(wvX, wvY)

			lineStyle = JWN_GetNumberFromWaveNote(wvY, SF_META_LINESTYLE)
			if(IsValidTraceLineStyle(lineStyle))
				ModifyGraph/W=$pg.win lStyle($trace)=lineStyle
			elseif(formulasAreDifferent)
				ModifyGraph/W=$pg.win lStyle($trace)=lineCode
			endif

			WAVE/Z customMarkerAsFree = JWN_GetNumericWaveFromWaveNote(wvY, SF_META_MOD_MARKER)
			if(WaveExists(customMarkerAsFree))
				DFREF dfrWork = SFH_GetWorkingDF(pg.graph)
				wvName = "customMarker_" + NameOfWave(wvY)
				WAVE customMarker = MoveFreeWaveToPermanent(customMarkerAsFree, dfrWork, wvName)
				ASSERT(DimSize(wvY, ROWS) == DimSize(customMarker, ROWS), "Marker size mismatch")
				ModifyGraph/W=$pg.win zmrkNum($trace)={customMarker}
			else
				overrideMarker = JWN_GetNumberFromWaveNote(wvY, SF_META_MOD_MARKER)

				if(!IsNaN(overrideMarker))
					markerCode = overrideMarker
				endif

				ModifyGraph/W=$pg.win marker($trace)=markerCode
			endif

			traceToFront = JWN_GetNumberFromWaveNote(wvY, SF_META_TRACETOFRONT)
			traceToFront = IsNaN(traceToFront) ? 0 : !!traceToFront
			if(traceToFront)
				ReorderTraces/W=$pg.win _front_, {$trace}
			endif

		endfor
	endfor
End

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The function documentation at line 2177 states "Copy plot meta data JSON properties from a source to a target wave" but the implementation only copies three specific properties (TRACECOLOR, TRACETOFRONT, LINESTYLE) rather than all plot metadata. The documentation should accurately describe what is copied, e.g., "Copy specific plot meta data properties (trace color, trace-to-front flag, and line style) from a source to a target wave".

Suggested change

	/// @brief Copy plot meta data JSON properties from a source to a target wave
	/// @brief Copy specific plot meta data properties (trace color, trace-to-front flag, and line style) from a source to a target wave

[Copilot]

The error message at line 2498 has a grammatical issue. "Expected numeric wave with a single number at dataset" should be "Expected a numeric wave with a single number at dataset" for proper grammar.

Suggested change

	SFH_ASSERT(IsNumericWave(data) && DimSize(data, ROWS) == 1 && numpnts(data) == 1, "Expected numeric wave with a single number at dataset " + num2istr(i))
	SFH_ASSERT(IsNumericWave(data) && DimSize(data, ROWS) == 1 && numpnts(data) == 1, "Expected a numeric wave with a single number at dataset " + num2istr(i))

[Copilot]

The variable 'i' is being reused outside of the loop scope where it was defined. At line 2645, 'i' is used to index into plotWITH after the for loop at lines 2612-2632 has completed. This relies on 'i' retaining the value 'numExp' after the loop ends. While this works in Igor Pro, it's a maintainability issue as the code depends on implicit loop behavior. It would be clearer to either use a new variable or explicitly set it to 'numExp' to make the intent obvious.

[Copilot]

There's duplicated logic for handling xaxisOffset and yaxisOffset. Lines 2575-2584 and 2648-2657 contain nearly identical if-elseif-else chains for xaxisOffset handling, while lines 2613-2621 and 2634-2642 duplicate the yaxisOffset handling. This violates DRY principles and makes maintenance harder. Consider extracting this logic into a helper function that takes the offset type, variable name, and returns the appropriate formula string.

[Copilot]

The function has inconsistent parameter naming. The parameters use 'xaxisOffset' and 'yaxisOffset' (lowercase 'axis'), but the percentage parameters use 'xAxisPercentage' and 'yAxisPercentage' (camelCase 'Axis'). For consistency, either all should use lowercase ('xaxis', 'yaxis') or all should use camelCase ('xAxis', 'yAxis').

Suggested change

	xAxisPercentage = SFH_GetArgumentAsNumeric(exd, opShort, 2, defValue = 100, checkFunc = BetweenZeroAndOneHoundred)
	yAxisPercentage = SFH_GetArgumentAsNumeric(exd, opShort, 3, defValue = 100, checkFunc = BetweenZeroAndOneHoundred)
	xaxisPercentage = SFH_GetArgumentAsNumeric(exd, opShort, 2, defValue = 100, checkFunc = BetweenZeroAndOneHoundred)
	yaxisPercentage = SFH_GetArgumentAsNumeric(exd, opShort, 3, defValue = 100, checkFunc = BetweenZeroAndOneHoundred)

[MichaelHuth]

@timjarsky Regarding the min, max, none behavior: It is implemented as specified in the original issue and min is not normalizing to the minimum but to the first point from the datasets. I did not remember that immediately in the discussion. This explains the negative numbers still present for the x-values.

min -> merge($data - extract($data, 0))
max -> merge($data- max(merge($data)))
none -> merge($data)

So maybe min should be renamed to e.g. first ?

[timjarsky]

@timjarsky Regarding the min, max, none behavior: It is implemented as specified in the original issue and min is not normalizing to the minimum but to the first point from the datasets. I did not remember that immediately in the discussion. This explains the negative numbers still present for the x-values.

min -> merge($data - extract($data, 0))
max -> merge($data- max(merge($data)))
none -> merge($data)

So maybe min should be renamed to e.g. first ?

Hi @MichaelHuth, is the first point in $data the first sweep acquired, or is it something else? Normalizing to the minimum would be helpful.

[MichaelHuth]

@timjarsky Regarding the min, max, none behavior: It is implemented as specified in the original issue and min is not normalizing to the minimum but to the first point from the datasets. I did not remember that immediately in the discussion. This explains the negative numbers still present for the x-values.

min -> merge($data - extract($data, 0))
max -> merge($data- max(merge($data)))
none -> merge($data)

So maybe min should be renamed to e.g. first ?

Hi @MichaelHuth, is the first point in $data the first sweep acquired, or is it something else? Normalizing to the minimum would be helpful.

The result from select that is relevant for the order is sorted by sweepNumber -> channelType -> channelNumber (in that priority). So e.g. extract($freq0, 0) results in the apfrequency of the sweep with the lowest sweep number of the selection.

I will change that option to:

first -> merge($data - extract($data, 0))
min -> merge($data - min(merge($data)))
max -> merge($data- max(merge($data)))
none -> merge($data)

[timjarsky]

@MichaelHuth , can you confirm that min -> merge($data - min(merge($data))) happens at the experiment level? Or, does it happen across all experiments (That is, the minimum current across all experiments is subtracted from each experiment)?

[MichaelHuth]

The min offset is determined for each experiment.

Here are the internal formulas for a three experiment example with ivscc_apfrequency(min, min, 100, 100, 0, -60):

sel = select(selsweeps(), selstimset("*rheo*", "*supra*"), selvis(all), selivsccsweepqc(passed))

selexpAD0 = select(selexp("Vip-IRES-Cre-835274.13.03.01.nwb"), $sel, selchannels(AD0), selrange(E1))
selexpDA0 = select(selexp("Vip-IRES-Cre-835274.13.03.01.nwb"), $sel, selchannels(DA0), selrange(E1))
freq0 = apfrequency(data($selexpAD0), 0, -60.000000, freq, nonorm, time)
current0 = max(data($selexpDA0))
currentNorm0 = $current0 - min(merge($current0))

selexpAD1 = select(selexp("Vip-IRES-Cre-835274.14.03.01.nwb"), $sel, selchannels(AD0), selrange(E1))
selexpDA1 = select(selexp("Vip-IRES-Cre-835274.14.03.01.nwb"), $sel, selchannels(DA0), selrange(E1))
freq1 = apfrequency(data($selexpAD1), 0, -60.000000, freq, nonorm, time)
current1 = max(data($selexpDA1))
currentNorm1 = $current1 - min(merge($current1))

selexpAD2 = select(selexp("Vip-IRES-Cre-835274.14.03.02.nwb"), $sel, selchannels(AD0), selrange(E1))
selexpDA2 = select(selexp("Vip-IRES-Cre-835274.14.03.02.nwb"), $sel, selchannels(DA0), selrange(E1))
freq2 = apfrequency(data($selexpAD2), 0, -60.000000, freq, nonorm, time)
current2 = max(data($selexpDA2))
currentNorm2 = $current2 - min(merge($current2))

ivsccavg = avg([$freq0,$freq1,$freq2], group)
ivscccurrentavg = avg([$currentNorm0,$currentNorm1,$currentNorm2], group)

ivscc_apfrequency_explist = ["Vip-IRES-Cre-835274.13.03.01.nwb","Vip-IRES-Cre-835274.14.03.01.nwb","Vip-IRES-Cre-835274.14.03.02.nwb"]

merge($freq0 - min(merge($freq0))) vs merge($currentNorm0)
with
merge($freq1 - min(merge($freq1))) vs merge($currentNorm1)
with
merge($freq2 - min(merge($freq2))) vs merge($currentNorm2)
with
merge($ivsccavg - min(merge($ivsccavg))) vs merge($ivscccurrentavg - min(merge($ivscccurrentavg)))

I have grouped the variables by adding a empty lines (need to be removed when pasted in a notebook).

[timjarsky]

@MichaelHuth min argument is looking good. I need to dig into the data for each experiment to see why there are an unexpectedly high number of zero freq points at high current values.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The comment states 'Sets the plot meta data for the ivscc_apfrequency operation' but the function only sets two specific properties (xAxisPercentage and yAxisPercentage). The comment should be more specific about what properties are being set rather than implying all plot metadata is being configured.

Suggested change

	/// @brief Sets the plot meta data for the ivscc_apfrequency operation
	/// @brief Sets the x- and y-axis percentage plot metadata for the ivscc_apfrequency operation

[MichaelHuth]

ivsccavg = avg([$freq0,$freq1,$freq2], bins, [0, 500], 100, [$currentNorm0,$currentNorm1,$currentNorm2])

@timjarsky I changed the logic using the new bins mode of avg that I added.

It works like described in issue #2581

The syntax is avg([dataGroups, ...], bins, [binsStart, binsEnd], binsWidth, [binGroups, ...])

The result is a wave with number of bins datasets. Number of dataGroups must equal number of binGroups and binGroups must have a single data point per dataset.

This looks in the internal formula used by ivscc_apfrequency then like this:

ivsccavg = avg([$freq0,$freq1,$freq2], bins, [0, 500], 100, [$currentNorm0,$currentNorm1,$currentNorm2])

For three experiments with a bin range from 0 to 500 and a bin size of 100 (pA in that case as the bins refer to the binGroups which are current)

I inserted the bin range and bin width argument after the first four arguments, such that:
ivscc_apfrequency(xaxisOffset, yaxisOffset, xaxisPercent, yaxisPercent, [binStart, binEnd], binWidth, [method, level, resultType, normalize, xaxisType])

The brackets around binStart, binEnd are required, the brackets around the apfrequency related arguments indicate that these are optional.

e.g.
ivscc_apfrequency(none, none, 100, 100, [0,600], 100, 0, -60)

I changed the x-formula for the average output to contain the bin centers.