• Skip to main content
  • Skip to header right navigation
  • Skip to site footer

My Online Training Hub

Learn Dashboards, Excel, Power BI, Power Query, Power Pivot

  • Courses
  • Pricing
    • Free Courses
    • Power BI Course
    • Excel Power Query Course
    • Power Pivot and DAX Course
    • Excel Dashboard Course
    • Excel PivotTable Course – Quick Start
    • Advanced Excel Formulas Course
    • Excel Expert Advanced Excel Training
    • Excel Tables Course
    • Excel, Word, Outlook
    • Financial Modelling Course
    • Excel PivotTable Course
    • Excel for Customer Service Professionals
    • Excel for Operations Management Course
    • Excel for Decision Making Under Uncertainty Course
    • Excel for Finance Course
    • Excel Analysis ToolPak Course
    • Multi-User Pricing
  • Resources
    • Free Downloads
    • Excel Functions Explained
    • Excel Formulas
    • Excel Add-ins
    • IF Function
      • Excel IF Statement Explained
      • Excel IF AND OR Functions
      • IF Formula Builder
    • Time & Dates in Excel
      • Excel Date & Time
      • Calculating Time in Excel
      • Excel Time Calculation Tricks
      • Excel Date and Time Formatting
    • Excel Keyboard Shortcuts
    • Excel Custom Number Format Guide
    • Pivot Tables Guide
    • VLOOKUP Guide
    • ALT Codes
    • Excel VBA & Macros
    • Excel User Forms
    • VBA String Functions
  • Members
    • Login
    • Password Reset
  • Blog
  • Excel Webinars
  • Excel Forum
    • Register as Forum Member

Improve speed on VBA loop to insert multiple column into MySQL|VBA & Macros|Excel Forum|My Online Training Hub

You are here: Home / Improve speed on VBA loop to insert multiple column into MySQL|VBA & Macros|Excel Forum|My Online Training Hub
Avatar
sp_LogInOut Log In sp_Registration Register
sp_Search Search
Advanced Search|Last Search Results
Search
Forum Scope




Match



Forum Options



Minimum search word length is 3 characters - maximum search word length is 84 characters
sp_Search Search
sp_RankInfo
Lost password?
sp_CrumbsHome HomeExcel ForumVBA & MacrosImprove speed on VBA loop to insert…
sp_PrintTopic sp_TopicIcon
Improve speed on VBA loop to insert multiple column into MySQL
Avatar
yhooithin05
Member
Members
Level 0
Forum Posts: 61
Member Since:
July 11, 2016
sp_UserOfflineSmall Offline
1
September 24, 2019 - 11:22 pm
sp_Permalink sp_Print

Hi,

I have a VBA loop that inserts around 50 rows with 6 columns into a MySQL table.

Everything runs well, however currently it take approx 20 seconds to insert those 50 rows (loop 50 times) ,this is far too much. I just want to know if there is a faster way (just one INSERT query in one hit), inserting all 50 rows, all at once but I don't know if it is possible. 

I have search all the solution around and mostly suggested to use bulk insert, but i do not know how to use it.

What I want is simplicity & super-fast uploading.

Please note that I am a beginner on this so if you can stir me into some samples on how it should be done it will be much appreciated.

Thanks guys!

Regards,

Hooi Thin

Avatar
yhooithin05
Member
Members
Level 0
Forum Posts: 61
Member Since:
July 11, 2016
sp_UserOfflineSmall Offline
2
September 24, 2019 - 11:36 pm
sp_Permalink sp_Print

macro1.PNGImage Enlarger

macro2.PNGImage Enlarger
I cannot attached the sql coding here so i attached in picture for my macro coding.

sp_PlupAttachments Attachments
  • sp_PlupImage macro1.PNG (185 KB)
  • sp_PlupImage macro2.PNG (158 KB)
Avatar
Philip Treacy
Admin
Level 10
Forum Posts: 1510
Member Since:
October 5, 2010
sp_UserOfflineSmall Offline
3
September 25, 2019 - 9:41 am
sp_Permalink sp_Print

Hi Hooi Thin,

I'm not sure why you can't add the SQL (it's just VBA in a workbook).

In your loop you are doing a SELECT, so you are doing that 50 times.  That will slow things down.

Regards

Phil

Avatar
yhooithin05
Member
Members
Level 0
Forum Posts: 61
Member Since:
July 11, 2016
sp_UserOfflineSmall Offline
4
September 25, 2019 - 11:41 am
sp_Permalink sp_Print

Hi Philip,

The reason i can't add the SQL here is because I get access denied - Sucuri Website Firewall (Block reason: SQL injection was detected and blocked.)

Yes, is there an alternative way to change the query execute once rather than 50 times, can you guide me please? I've been struggling this for several days 🙁

Thanks.

Regards,

Hooi Thin

Avatar
Philip Treacy
Admin
Level 10
Forum Posts: 1510
Member Since:
October 5, 2010
sp_UserOfflineSmall Offline
5
September 26, 2019 - 10:20 am
sp_Permalink sp_Print

So attaching a workbook with VBA (SQL) in it to your forum comment gives you an error?  Have you clicked on the Attachments button below and tried adding/uploading a workbook?

With regards to your SQL, why are you doing a SELECT every loop?  It looks like what you are trying to do is look for a specific record and if it isn't found you do the INSERT.

However, what it is actually doing is looking through every row in your worksheet, checking (SELECT) the database to see if those values are already in there, BUT because your INSERT statement is outside the FOR loop, the code is only doing 1 INSERT and this is for the last record not found to match your data in the workbook.

So it's not doing 50 INSERTS the code is only doing 1 at most every time you run the VBA Sub.  As I'm not clear on what it is you are trying to do exactly, and I haven't seen your data or database, I can't get a lot more specific in my advice.

Of course removing the SELECT will speed things up but I don't know if you need it there or not.

If you can explain what you are trying to do, and what you expect the code to do, and supply the workbook with data, I can probably help more 🙂

Cheers

Phil

Avatar
yhooithin05
Member
Members
Level 0
Forum Posts: 61
Member Since:
July 11, 2016
sp_UserOfflineSmall Offline
6
September 26, 2019 - 12:45 pm
sp_Permalink sp_Print

Hi Philip,

I did't try to upload workbook with SQL but pasted macro with SQL coding at here having the error.

Yes you are right, I am doing the SELECT statement inside for loop just to checking if each row in excel exist in MySQL db, if not found then will insert so i need the SELECT.

INSERT statement is outside the FOR loop --  previously i put in for loop, but i move out after that to populate string inside the loop just to prepare a complete full query, after full query done then make one time insert query to execute to database. I thought this way will be faster but its not.

Finally I have found the root cause, I am currently running this vba on my local country Malaysia, but the server itself is remote on US.  i tested to import to my local db, 1 second can insert but to remote server need longer time, i think the best way is not to use VBA as it have limitation on time constraint issue? If want to stay at VBA what can i do?

Thanks!

Regards,

Hooi Thin

Avatar
Philip Treacy
Admin
Level 10
Forum Posts: 1510
Member Since:
October 5, 2010
sp_UserOfflineSmall Offline
7
September 27, 2019 - 9:27 am
sp_Permalink sp_Print

I think the issue is more to do with the DB being in the USA than you using VBA.  If you are doing database operations remotely like that you must expect delays like you are experiencing.  Working on a DB on your local machine will always be faster.  So, if you have to work with the database being remote like this you may have to put up with it.

I don't know what type of server your DB is stored on.  If it's a low powered machine you may get better response by using something with more power.  You could look at Azure.  Do you have to use MySQL?  Are you using this because it's hosted on a web server?

If you want to try getting away from VBA you could look at MySQL Workbench but I don't know what performance improvements that will give, if any, over your current setup.

Regards

Phil

Avatar
yhooithin05
Member
Members
Level 0
Forum Posts: 61
Member Since:
July 11, 2016
sp_UserOfflineSmall Offline
8
September 27, 2019 - 12:58 pm
sp_Permalink sp_Print

Hi Phil,

I think i will continue with VBA and put up with it. I need to use MySQL as all of my application are hosted there.

Our server is on virtual platform with microsoft windows server 2012 R2 Standard.

After all, I think still limitation i can do as all things control by our organisation.

Thanks a lot for your advise and guidance! Appreciate that 🙂

Regards,

Hooi Thin

sp_Feed
Go to top
Forum Timezone: Australia/Brisbane
Most Users Ever Online: 245
Currently Online: Jessica Stewart, Mark Carlson, Calvin Richardson, Debi Morgan
Guest(s) 5
Currently Browsing this Page:
1 Guest(s)
Top Posters:
SunnyKow: 1432
Anders Sehlstedt: 870
Purfleet: 412
Frans Visser: 346
David_Ng: 306
lea cohen: 219
A.Maurizio: 202
Jessica Stewart: 202
Aye Mu: 201
jaryszek: 183
Newest Members:
Debi Morgan
yashal minahil
Oluwadamilola Ogun
Yannik H
dectator mang
Francis Drouillard
Orlando Inocente
Jovitha Clemence
Maloxat Axmatovna
Ricardo Freitas
Forum Stats:
Groups: 3
Forums: 24
Topics: 6201
Posts: 27185

 

Member Stats:
Guest Posters: 49
Members: 31861
Moderators: 3
Admins: 4
Administrators: Mynda Treacy, Philip Treacy, Catalin Bombea, FT
Moderators: MOTH Support, Velouria, Riny van Eekelen
© Simple:Press —sp_Information

Sidebar

Blog Categories

  • Excel
  • Excel Charts
  • Excel Dashboard
  • Excel Formulas
  • Excel PivotTables
  • Excel Shortcuts
  • Excel VBA
  • General Tips
  • Online Training
  • Outlook
  • Power Apps
  • Power Automate
  • Power BI
  • Power Pivot
  • Power Query
microsoft mvp logo
trustpilot excellent rating
Secured by Sucuri Badge
MyOnlineTrainingHub on YouTube Mynda Treacy on Linked In Mynda Treacy on Instagram Mynda Treacy on Twitter Mynda Treacy on Pinterest MyOnlineTrainingHub on Facebook
 

Company

  • About My Online Training Hub
  • Disclosure Statement
  • Frequently Asked Questions
  • Guarantee
  • Privacy Policy
  • Terms & Conditions
  • Testimonials
  • Become an Affiliate

Support

  • Contact
  • Forum
  • Helpdesk - For Technical Issues

Copyright © 2023 · My Online Training Hub · All Rights Reserved. Microsoft and the Microsoft Office logo are trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Product names, logos, brands, and other trademarks featured or referred to within this website are the property of their respective trademark holders.