I am receiving a response as attached in the image. So i am getting an individua record as an array. So I used flatten on rows column. After that i used index function in derived column activity (e.g rows[0],rows2) to extract each record.

I have highlighted the array, that i am getting as a whole record to a single column (e.g i used rows[19]) to extract to column called Tags.

So my target is i want to separate that array record to again individual record against each parent array record as i mentioned in the table. In the table Parent Array data 1 is having tag value, similarly another array is also having tag value as array record.

I hope i have explained what i am trying to achieve. I will explain bit more, if it is clear.

I have explained in the previous section.

I am trying to extract the data from Azure Cost management API for my organisation. I have removed the record within the inverted double quotes for privacy reason.

This is my dataflow json : { "name": "df_finops", "properties": { "type": "MappingDataFlow", "typeProperties": { "sources": [ { "dataset": { "referenceName": "ds_rest_finops_dev", "type": "DatasetReference" }, "name": "apiAzureCostManagement" } ], "sinks": [ { "linkedService": { "referenceName": "ls_adls_gen2", "type": "LinkedServiceReference" }, "name": "FinopsADLS" } ], "transformations": [ { "name": "flattenRowColumn" }, { "name": "derivedColumnExtractingRowData" }, { "name": "selectRequiredColumns" }, { "name": "derivedColumnFolderMapping" } ], "scriptLines": [ "parameters{", " OauthToken as string,", " FromTime as string,", " ToTime as string", "}", "source(output(", " body as (eTag as string, id as string, location as string, name as string, properties as (columns as (name as string, type as string)[], nextLink as string, rows as string[][]), sku as string, type as string),", " headers as [string,string]", " ),", " allowSchemaDrift: true,", " validateSchema: false,", " httpMethod: 'POST',", " body: (concat('{"type":"Usage","timeframe":"Custom","timePeriod":{"from":"',$FromTime,'","to":"',$ToTime,'"},"dataset":{"granularity":"Daily","aggregation":{"totalCost":{"name":"Cost","function":"Sum"}},"grouping":[{"type":"Dimension","name":"ResourceGroupName"},{"type":"Dimension","name":"ResourceType"},{"type":"Dimension","name":"MeterCategory"},{"type":"Dimension","name":"ServiceName"},{"type":"Dimension","name":"ChargeType"},{"type":"Dimension","name":"InvoiceId"},{"type":"Dimension","name":"Frequency"},{"type":"Dimension","name":"Product"},{"type":"Dimension","name":"ProductOrderId"},{"type":"Dimension","name":"Provider"},{"type":"Dimension","name":"PublisherType"},{"type":"Dimension","name":"ResourceId"},{"type":"Dimension","name":"ResourceGuid"},{"type":"Dimension","name":"ServiceFamily"},{"type":"Dimension","name":"BillingMonth"}],"include": ["Tags"]}}')),", " timeout: 30,", " requestInterval: 0,", " headers: ['Authorization' -> (concat('Bearer ',$OauthToken)), 'Content-Type' -> ('application/json')],", " paginationRules: ['absoluteUrl' -> 'body.properties.nextLink'],", " responseFormat: ['type' -> 'json', 'documentForm' -> 'arrayOfDocuments']) ~> apiAzureCostManagement", "apiAzureCostManagement foldDown(unroll(body.properties.rows),", " mapColumn(", " columns = body.properties.columns,", " rows = body.properties.rows", " ),", " skipDuplicateMapInputs: false,", " skipDuplicateMapOutputs: false) ~> flattenRowColumn", "flattenRowColumn derive(Cost = rows1,", " UsageDate = rows2,", " ResourceGroupName = rows3,", " ResourceType = rows4,", " MeterCategory = rows[5],", " ServiceName = rows[6],", " ChargeType = rows[7],", " InvoiceId = rows[8],", " Frequency = rows[9],", " Product = rows[10],", " ProductOrderId = rows[11],", " Provider = rows[12],", " PublisherType = rows[13],", " ResourceId = rows[14],", " ResourceGuid = rows[15],", " ServiceFamily = rows[16],", " BillingMonth = rows[17],", " Tags = rows[18],", " Currency = rows[19]) ~> derivedColumnExtractingRowData", "derivedColumnFolderMapping select(mapColumn(", " Cost,", " UsageDate,", " ResourceGroupName,", " ResourceType,", " MeterCategory,", " ServiceName,", " ChargeType,", " InvoiceId,", " Frequency,", " Product,", " ProductOrderId,", " Provider,", " PublisherType,", " ResourceId,", " ResourceGuid,", " ServiceFamily,", " BillingMonth,", " Tags,", " Currency,", " year,", " month,", " day", " ),", " skipDuplicateMapInputs: true,", " skipDuplicateMapOutputs: true) ~> selectRequiredColumns", "derivedColumnExtractingRowData derive(year = year(toDate(BillingMonth)),", " month = month(toDate(BillingMonth)),", " day = dayOfMonth(toDate(BillingMonth))) ~> derivedColumnFolderMapping", "selectRequiredColumns sink(allowSchemaDrift: true,", " validateSchema: false,", " format: 'parquet',", " fileSystem: 'finops',", " folderPath: 'costmanagement',", " compressionCodec: 'snappy',", " filePattern:'azurecost',", " umask: 0007,", " preCommands: [],", " postCommands: [],", " skipDuplicateMapInputs: true,", " skipDuplicateMapOutputs: true,", " partitionBy('key',", " 0,", " year,", " month,", " day", " )) ~> FinopsADLS" ] } } }