• Skip to main content
  • Skip to header right navigation
  • Skip to site footer

My Online Training Hub

Learn Dashboards, Excel, Power BI, Power Query, Power Pivot

  • Courses
  • Pricing
    • Free Courses
    • Power BI Course
    • Excel Power Query Course
    • Power Pivot and DAX Course
    • Excel Dashboard Course
    • Excel PivotTable Course โ€“ Quick Start
    • Advanced Excel Formulas Course
    • Excel Expert Advanced Excel Training
    • Excel Tables Course
    • Excel, Word, Outlook
    • Financial Modelling Course
    • Excel PivotTable Course
    • Excel for Customer Service Professionals
    • Excel for Operations Management Course
    • Excel for Decision Making Under Uncertainty Course
    • Excel for Finance Course
    • Excel Analysis ToolPak Course
    • Multi-User Pricing
  • Resources
    • Free Downloads
    • Excel Functions Explained
    • Excel Formulas
    • Excel Add-ins
    • IF Function
      • Excel IF Statement Explained
      • Excel IF AND OR Functions
      • IF Formula Builder
    • Time & Dates in Excel
      • Excel Date & Time
      • Calculating Time in Excel
      • Excel Time Calculation Tricks
      • Excel Date and Time Formatting
    • Excel Keyboard Shortcuts
    • Excel Custom Number Format Guide
    • Pivot Tables Guide
    • VLOOKUP Guide
    • ALT Codes
    • Excel VBA & Macros
    • Excel User Forms
    • VBA String Functions
  • Members
    • Login
  • Blog
  • Excel Webinars
  • Excel Forum
    • Register as Forum Member
  • Login

Multiple csv-files and changing content|Power Query|Excel Forum|My Online Training Hub

You are here: Home / Multiple csv-files and changing content|Power Query|Excel Forum|My Online Training Hub
Avatar
sp_LogInOut Log In sp_Registration Register
sp_Search Search
Advanced Search|Last Search Results
Search
Forum Scope




Match



Forum Options



Minimum search word length is 3 characters - maximum search word length is 84 characters
sp_Search Search
sp_RankInfo
Lost password?
sp_CrumbsHome HomeExcel ForumPower QueryMultiple csv-files and changing con…
sp_PrintTopic sp_TopicIcon
Multiple csv-files and changing content
Avatar
Maarten de Keijzer
Member
Members

Power Pivot
Level 0
Forum Posts: 35
Member Since:
September 2, 2020
sp_UserOfflineSmall Offline
1
October 8, 2020 - 5:51 pm
sp_Permalink sp_Print

Hi,

I like your examples with PowerQuery combining multiple csv-files and creating great pivots and graphs.

We started to rebuild our PerformancePortal with these techniques.

What is the best way to implement a change in file content? For instance: at some moment we want/need to have extra columns to show specific information. Is it possible to include newer columns into the 'merged' data model without having to change all previous csv-files?

ย 

kind regards, Maarten

sp_AnswersTopicSeeAnswer See Answer
Avatar
Catalin Bombea
Iasi, Romania
Admin
Level 10
Forum Posts: 1844
Member Since:
November 8, 2013
sp_UserOfflineSmall Offline
2
October 9, 2020 - 1:12 am
sp_Permalink sp_Print

Hi Maarten,

See this post for a solution, download that file attached: https://www.myonlinetraininghu.....der#p16593

Avatar
Maarten de Keijzer
Member
Members

Power Pivot
Level 0
Forum Posts: 35
Member Since:
September 2, 2020
sp_UserOfflineSmall Offline
3
October 9, 2020 - 11:39 pm
sp_Permalink sp_Print

Thanks for the response.

But: apparently it will not or cannot work for files from a SharePoint map. Do you have a likewise solution for that environment?

And I suspect that this solution deals with the adding of new files, but my real question was about file content.

For instance we have a lot of monthly files with project data, and starting from this month we want to register the responsabel account manager,

that will be registered in a new column.

Avatar
Catalin Bombea
Iasi, Romania
Admin
Level 10
Forum Posts: 1844
Member Since:
November 8, 2013
sp_UserOfflineSmall Offline
4
October 9, 2020 - 11:46 pm
sp_Permalink sp_Print

What is a SharePoint Map? Is that a sharepoint site documents?

If yes, you can simply Sync that folder locally, and you will always have files syncronized with sharepoint site. Use the local synced folder as source for extracting data.

Avatar
Maarten de Keijzer
Member
Members

Power Pivot
Level 0
Forum Posts: 35
Member Since:
September 2, 2020
sp_UserOfflineSmall Offline
5
October 12, 2020 - 4:59 pm
sp_Permalink sp_Print

The power of using files from a SharePoint folder is that everyonbe with SharePoint rights can update the file; that's not possible when we use a synced folder.

But still no answer to my question: how to deal with new (necessary) columns.

Avatar
Catalin Bombea
Iasi, Romania
Admin
Level 10
Forum Posts: 1844
Member Since:
November 8, 2013
sp_UserOfflineSmall Offline
6
October 12, 2020 - 5:17 pm
sp_Permalink sp_Print

Actually, the link I provided offers a solution where it does not matter how many columns are in the source files, so you do have an answer. My question in fact does not have an answer ๐Ÿ™‚ ("What is a SharePoint Map?")

All you have to do is to change the query to get data from sharepoint folder instead of local folder. The function that reads the files will consider any new columns added.

Avatar
Catalin Bombea
Iasi, Romania
Admin
Level 10
Forum Posts: 1844
Member Since:
November 8, 2013
sp_UserOfflineSmall Offline
7
October 12, 2020 - 5:21 pm
sp_Permalink sp_Print

The power of using files from a SharePoint folder is that everyonbe with SharePoint rights can update the file; that's not possible when we use a synced folder.

It's about how we read the data, not how we EDIT files.
Syncing the sharepoint folder with your desktop will only change how you READ the files: you read them from the local folder, instead of reading from web.
OneDrive Sync agent will take care of bringing the online changes to your computer, so I did not suggested editing the source files OFFLINE.

Avatar
Maarten de Keijzer
Member
Members

Power Pivot
Level 0
Forum Posts: 35
Member Since:
September 2, 2020
sp_UserOfflineSmall Offline
8
October 12, 2020 - 9:15 pm
sp_Permalink sp_Print

I'm sorry for the misunderstandings

  1. in Dutch a 'map' stands for a 'folder', so I mean the csv-files are in a SharePoint folder
  2. with "everyone with SharePoint rights can update the file" I mean 'update the data' within the Excel file with the Power Pivot and graphs (also located in SharePoint) , not the underlying cvs-files
  3. "The function that reads the files will consider any new columns added"; I just tried that, but the extra columns in the newest csv-files didn't show up, so what function do you mean?
Avatar
Catalin Bombea
Iasi, Romania
Admin
Level 10
Forum Posts: 1844
Member Since:
November 8, 2013
sp_UserOfflineSmall Offline
9
October 13, 2020 - 12:27 am
sp_Permalink sp_Print

Please upload a few sample csv files with different headers and just 1-2 rows of fake data.
Will prepare a sample.

Avatar
Maarten de Keijzer
Member
Members

Power Pivot
Level 0
Forum Posts: 35
Member Since:
September 2, 2020
sp_UserOfflineSmall Offline
10
October 14, 2020 - 10:23 pm
sp_Permalink sp_Print

I have some data csv files and a simple Excel sheet to read them.

The first 3 (folder Data) are the 'standard' ones, in the folder NewData I added one with extra columns.

In the Excel sheet I got the data of the first 3 by 'reading csv from folder'.

After making pivot table I placed the 4th file and updates all data, but no extra columns.

Avatar
Philip Treacy
Admin
Level 10
Forum Posts: 1549
Member Since:
October 5, 2010
sp_UserOfflineSmall Offline
11
October 14, 2020 - 10:31 pm
sp_Permalink sp_Print

Hi Maarten,

Did you mean to attach files? You need to click the Start Upload button after selecting the files.

Phil

Avatar
Maarten de Keijzer
Member
Members

Power Pivot
Level 0
Forum Posts: 35
Member Since:
September 2, 2020
sp_UserOfflineSmall Offline
12
October 15, 2020 - 12:42 am
sp_Permalink sp_Print

Sorry, I didn't notice that!

Avatar
Catalin Bombea
Iasi, Romania
Admin
Level 10
Forum Posts: 1844
Member Since:
November 8, 2013
sp_UserOfflineSmall Offline
13
October 15, 2020 - 1:29 am
sp_Permalink sp_Print

Using the automatic file combiner is not dynamic. It ONLY reads the first file in that folder to get the headers and ignore all other headers from ALL other files.

Just use the file I already provided.

File attached here in case you don't want to download it from the other post i indicated.

Note that I had to change the csv delimiter, in your csv's the delimiter is semicolon instead of comma.

Avatar
Maarten de Keijzer
Member
Members

Power Pivot
Level 0
Forum Posts: 35
Member Since:
September 2, 2020
sp_UserOfflineSmall Offline
14
October 15, 2020 - 8:56 pm
sp_Permalink sp_Print

OK, thanks!
I will use it in our sheets.

And yes: in Dutch regional settings we always have to change things (. -> ,ย  and , -> ๐Ÿ˜‰

sp_AnswersTopicAnswer
Answers Post
sp_Feed
Go to top
Forum Timezone: Australia/Brisbane
Most Users Ever Online: 245
Currently Online: Tracy English
Guest(s) 9
Currently Browsing this Page:
1 Guest(s)
Top Posters:
SunnyKow: 1432
Anders Sehlstedt: 880
Purfleet: 414
Frans Visser: 346
David_Ng: 306
lea cohen: 237
Jessica Stewart: 219
A.Maurizio: 213
Aye Mu: 201
Hans Hallebeek: 183
Newest Members:
Rob Rooth
Tom Lewis
Jennifer Rodriguez-Avila
Khaled Ibrahim
Kiran Supekar
Lisa Myers
Ronald White
Ginette Guevremont
Taryn Ambrosi
Mark Davenport
Forum Stats:
Groups: 3
Forums: 24
Topics: 6531
Posts: 28604

 

Member Stats:
Guest Posters: 49
Members: 32822
Moderators: 2
Admins: 4
Administrators: Mynda Treacy, Philip Treacy, Catalin Bombea, FT
Moderators: Velouria, Riny van Eekelen
© Simple:Press —sp_Information

Sidebar

Blog Categories

  • Excel
  • Excel Charts
  • Excel Dashboard
  • Excel Formulas
  • Excel Office Scripts
  • Excel PivotTables
  • Excel Shortcuts
  • Excel VBA
  • General Tips
  • Online Training
  • Outlook
  • Power Apps
  • Power Automate
  • Power BI
  • Power Pivot
  • Power Query
microsoft mvp logo
trustpilot excellent rating
Secured by Sucuri Badge
MyOnlineTrainingHub on YouTube Mynda Treacy on Linked In Mynda Treacy on Instagram Mynda Treacy on Twitter Mynda Treacy on Pinterest MyOnlineTrainingHub on Facebook

Sign up to our newsletter and join over 400,000
others who learn Excel and Power BI with us.

 

Company

  • About My Online Training Hub
  • Disclosure Statement
  • Frequently Asked Questions
  • Guarantee
  • Privacy Policy
  • Terms & Conditions
  • Testimonials
  • Become an Affiliate
  • Sponsor Our Newsletter

Support

  • Contact
  • Forum
  • Helpdesk - For Technical Issues

Copyright © 2023 ยท My Online Training Hub ยท All Rights Reserved. Microsoft and the Microsoft Office logo are trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Product names, logos, brands, and other trademarks featured or referred to within this website are the property of their respective trademark holders.