Total Page Preview:   000000004296

What is Datasets in Azure Data Factory

In this article we will discussed about Datasets In Azure Data Factory.

Datasets:

Datasets represent data structures within the data stores, which simply points the data you want to use in your activities as inputs or outputs. An activity takes a zero or more datasets as inputs and one or more datasets as outputs. Example, an Azure Blob dataset specifies the blob container and folder in the Azure Blob Storage from which the pipeline should read the data. Or, an Azure SQL Table dataset specifies the table to which the output data is written by the activity.

Create Datasets in Azure Data Factory using portal :

 
 
{
    "name": "SourceAzureBlobDataset",
    "properties": {
        "structure": [
            {
                "name": "ProductId",
                "type": "Int64"
            },
            {
                "name": "ProductName",
                "type": "String"
            },
            {
                "name": "Quantity",
                "type": "Int64"
            },
            {
                "name": "City",
                "type": "String"
            }
        ],
        "published": false,
        "type": "AzureBlob",
        "linkedServiceName": "SourceAzureStorageLinkedService",
        "typeProperties": {
            "fileName": "Products.txt",
            "folderPath": "products",
            "format": {
                "type": "TextFormat",
                "columnDelimiter": ",",
                "firstRowAsHeader": true
            }
        },
        "availability": {
            "frequency": "Minute",
            "interval": 15
        },
        "external": true,
        "policy": {}
    }
}

 

Thank You

About Author

Brijesh Kumar

Database Developer

I have more then 6 years Experience in Microsoft Technologies - SQL Server Database, ETL Azure Cloud - Azure SQL Database, CosmosDB, Azure Data Factory, PowerBI, Web Job, Azure Function, Azure Storage, Web Apps, Powershall and Database Migration On-Premise to Azure Cloud.
LinkedIn : https://www.linkedin.com



Comments

Renfro
06-Nov-2018
Thanks for sharing your thoughts on Datasets in Azure Data Factory. Regards
Feaster
07-Feb-2019
Howdy! I'm at work surfing around your blog from my new iphone! Just wanted to say I love reading through your blog and look forward to all your posts! Carry on the superb work!
Mahlum
19-Oct-2021
Hi there! This is kind of off topic buut I need some advice from an established blog. Is it hard to set up your own blog? I'm not very techincal but I can figure things out pretty quick. I'm thinking about creating my own but I'm not sure here to begin. Do you have any ideas or suggestions? Cheers
Pederson
05-Nov-2021
Do you have a spam issue on this site; I also am a blogger, and I was wanting to know your situation; we have developed some nice methods and we are looking to trade techniques with others, please shoot me an e-mail if interested.

                           
                           

                           

                           

Facebook User: