Search code examples
databaseazureblobmissing-data

Blob File Not Appearing in Query Editor

I am new to Azure. Currently I wanted to learn about database creation, but I stumbled upon an issue regarding a blob file that I had uploaded not appearing in the query editor (the tutorial I watched does appear in the query editor).

I've searched for the solution, and I did several things that the solutions had mentioned:

  1. I've changed container access status from private to container
  2. I've added SQL database firewall to add current IP and allow Azure service to access server
  3. I've set up Azure AD to grant full access to my account
  4. I've checked the DB for source and sink are the same (I only create 1 DB)

Any help would greatly be appreciated, thanks

Anonymous Level Access to Container:

1

Uploaded File in Blob:

2

File Not Appearing in Query Editor:

3

File Not Appearing in Sink Destination:

4

Solution

  • There are two things that you need to consider about the target table while copying data to SQL database table.

    1. If target table should be created manually:

      but I stumbled upon an issue regarding a blob file that I had uploaded not appearing in the query editor (the tutorial I watched does appear in the query editor)

      In the tutorial, the target table was created manually through SQL query after uploading the blob file to storage account.

      The uploading Blob file has nothing to do with SQL table. The SQL table should be created manually, or it will be created automatically from the blob file after pipeline run.
      To get the table in Query editor pane or while creating ADF SQL table dataset, you need to create the respective table first with required structure.


      enter image description here
      After executing the query, click on refresh button and you can see your table. You can see the same table in the dataset, and you can refresh here as well.
      enter image description here

    2. If target table should be created automatically from the pipeline run:
      Go to your SQL table dataset and click on edit.


      enter image description here

      Here, you can give the desired table name that you want to create automatically. In copy activity sink, select Auto create table.


      enter image description here
      Now, your table will be created automatically after executing the pipeline.

    Reference:

    https://learn.microsoft.com/en-us/azure/data-factory/tutorial-copy-data-portal