![Day[9]TV](/images/header-logo.png){kind=logo}
✅ Automation: Stems from Audio Looper
Phillip Music DK30 Fall 2022 1 1
Description
🤖 RISE OF THE MACHINES 🤖
I am going to continue a project developed to rip stems from an audio looper when added sequentially. This project has reached the most important milestones already and is semi-automated however some improvements in it’s robustness and versatility remain a requested feature and goal.
I’d like it to run even-more-automated rather than almost-automated. I may develop it so that it automatically posts updates to this channel as well. The results will be a “short hand” version of an audio mixed down from a long overlapping loop. There are many ways to listen to these “short hand” versions I will play around with mixing sound.
There will be expected troubles in the looper itself so some loops will be less quality. I’m not going to explain everything fully.
I will be using: +Audacity +RIPX +AutoHotkey +FL Studio +Other software as practiced
This project will run in parallel with the automated art project here https://day9.tv/dk30/project/63446b83499cfd1e8e37fe39. They’re both automated projects. #RiseOfTheMachines
Recent Updates
So I decided what was polished enough to share was the MACROS made in Audacity. The other half of this is AutoHotkey but it is still in different phases. I decided not to share.
I’ve done my best to create a visual representation of steps done by the following as well as in-between steps.
The following files Macros can be used in Audacity. I did change some MACRO names so they are not ummm directly usabvle honestly. I did that so they made more sense for posting here rather than their original names.
These Audacity MACRO files are:
- 0_PrepareTheLopedSection.txt
- 1_UseLoopedLabelsToCrop.bct
- 2_ReorderNameBasicTracksVocalOriginalLoop.txt
- 3_LabelVocals.txt
- 4_Fir5tLayerRepeatAndlnvertLoop_SetupConfiguration.txt
- 5_Repeatable_AutoPrefixToMoveToLabelFirst.txt
- 5_Repeatab|e_XManualToTheNumberOfVocalLabels.txt
- 5_Repeatable_XManualWithoutAddedTrackForMoreControl.tXt
- 6_GetStreamVersisonsAndLoopedStemsAndExportAsThreeTypesToFileLocation.txt
- XXX_Utility_Zooming.txt
Manually prepare the looped section. After splitting the part where the piano stops and the first ending where the loop stops you use this tocreate a workspace for manually finding the looped section within.
0_PrepareTheLopedSection.txt
SnapToOff:
Copy:
NewStereoTrack:
Paste:
Invert:Use_Preset="
And then you use this one to cut the loop from the segment and manually check using the third layer.
1_UseLoopedLabelsToCrop.bct
SelectNone: LastTrack: PrevTrack: Toggle: MoveToNextLabel: FirstTrack: NextTrack: Toggle: Split: SelectNone: LastTrack: Toggle: MoveToNextLabel: FirstTrack: NextTrack: Toggle: Split: SelectNone: FirstTrack: NextTrack: Toggle: CursTrackStart: SelNextClip: SplitDelete: CursTrackEnd: SelPrevClip: SplitNew: SelectNone: FirstTrack: NextTrack: Toggle: CursTrackEnd: SelPrevClip: Repeat:Count=“1” TrackSolo: FirstTrack: Toggle: UnmuteTracks: SelectNone: FirstTrack: NextTrack: Toggle: CursTrackEnd: SelPrevClip: CursorLongJumpLeft: DefaultPlayStop:
And then… Reorder the names.
2_ReorderNameBasicTracksVocalOriginalLoop.txt
FitInWindow: FitV: SelectNone: FirstTrack: Toggle: SetTrack:Name=“OriginalStream0” Toggle: NextTrack: Toggle: SetTrack:Name=“PianoBackTrack” Toggle: NextTrack: Toggle: SetTrack:Name=“VocalExtraction” SelectNone: LastTrack: Toggle: TrackMoveTop: SelectNone: LastTrack: Toggle: CursTrackEnd: SelPrevClip:
THOSE WERE MANUAL STEPS (ABOVE) TO PREPARE THE AUDACITY FILES FOR AUTOMATION. MORE WORK TO AUTOMATE FINDING THE LOOP BEING DONE BY ANOTHER INDIVIDUAL. AT THIS POINT THE AUTOHOTKEY CAN TAKE THE FILE AND DO THE REST
And then… Label the vocals. You have to manually assign the number. This assignment is done by checking the loooped segment length an d inpouting it ther.
3_LabelVocals.txt
FitInWindow: FitV: Macro_001zOOM: SelectNone: FirstTrack: Toggle: SelEnd: SelStart:
And then… Invert the first loop and repeat it a large amount of times. Mix and Render the two tracks so you remove the sound from the inverted loop.
4_Fir5tLayerRepeatAndlnvertLoop_SetupConfiguration.txt
SelectNone: LastTrack: TrackMoveTop: SelectNone: FirstTrack: MoveToNextLabel: CursorShortJumpRight: StoreCursorPosition: SelectNone: LastTrack: Toggle: CursTrackEnd: SelPrevClip: SelPrevClip: Copy: Repeat:Count=“50” FitV: ShiftUp: MixAndRenderToNewTrack: SelectNone: FitInWindow: FitV: Macro_001zOOM:
And then… There’s two in this next segment. One adjusts the cursor position to loop across the labels for the automation. Otherwise it can be done without the first Auto text here and manually by the userr to determine the stem starting point. This is the noticable part I’ve been sharing in the past because it is visible.
5_Repeatable_AutoPrefixToMoveToLabelFirst.txt
CursProjectEnd: SelCursorStoredCursor: MoveToNextLabel: StoreCursorPosition: Macro_3Rip@Selection:
5_Repeatab|e_XManualToTheNumberOfVocalLabels.txt
Split:
NewStereoTrack:
Paste:
SnapToNearest:
CursorLeft:
Align_EndToSelEnd:
LastTrack:
SelStart:
CursorLeft:
SelNextClip:
SelCursorToNextClipBoundary:
PrevTrack:
Toggle:
Split:
LastTrack:
Toggle:
TrackClose:
SplitNew:
Toggle:
LastTrack:
Invert:Use_Preset="
! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !
And this one is for MANUAL ONLY - in case there is a loop that has a smaller section of the looped segment before vocals. This is NOT DONE BY THE AUTOMATION and allows finner detail.
5_Repeatable_XManualWithoutAddedTrackForMoreControl.tXt
Split: NewStereoTrack: Paste: SnapToNearest: CursorLeft: Align_EndToSelEnd: LastTrack: SelStart: CursorLeft: SelNextClip: SelCursorToNextClipBoundary: PrevTrack: Toggle: Split: LastTrack: Toggle: TrackClose:
And then… The grand finish. Now that all the stems are in place. Just have to do some fancy footwork of moving tracks, splitting sections, and exporting to folder locations.
*[ I captured this in more detail than prior. When using macros Audacity will not always update the visual if it is not a function to change to the visual directly it would seem from how this works. ] *
6_GetStreamVersisonsAndLoopedStemsAndExportAsThreeTypesToFileLocation.txt
Save:
SelectNone:
FirstTrack:
NextTrack:
NextTrack:
Toggle:
CursProjectEnd:
SelPrevClip:
CursorLongJumpLeft:
Toggle:
NextTrack:
Toggle:
SelPrevClip:
SelPrevClip:
PrevTrack:
Toggle:
Split:
SelPrevClip:
Split:
SelPrevClip:
Split:
SelPrevClip:
Split:
SelPrevClip:
Split:
SelPrevClip:
Split:
SelPrevClip:
Split:
SelPrevClip:
Split:
Macro_001_ExportMiniLoops:
Toggle:
SetTrack:Name=“OriginalLoop1”
Align_StartToZero:
SelTrackStartToEnd:
Repeat:Count=“1”
Macro_001_ExportMiniLoops:
Toggle:
SetTrack:Name=“OriginalLoop2”
Align_StartToZero:
SelTrackStartToEnd:
Repeat:Count=“1”
Macro_001_ExportMiniLoops:
Toggle:
SetTrack:Name=“OriginalLoop3”
Align_StartToZero:
SelTrackStartToEnd:
Repeat:Count=“1”
UnmuteAllTracks:
TrackSolo:
UnmuteAllTracks:
SelectNone:
LastTrack:
Toggle:
SelTrackStartToEnd:
TrackSolo:
Export2:Filename=“C:\Users\Admin\Music\AudacityEXPORTS\SingleLoopDump\\OriginalLoopCompare\WAV_Unedit\Unedit_exported3.wav” NumChannels=“2”
Normalize:ApplyGain=“1” PeakLevel="-1" RemoveDcOffset=“1” StereoIndependent=“0”
Export2:Filename=“C:\Users\Admin\Music\AudacityEXPORTS\SingleLoopDump\OriginalLoopCompare\WAV_Normalized\Normalized_exported3.wav” NumChannels=“2”
Export2:Filename=“C:\Users\Admin\Music\AudacityEXPORTS\SingleLoopDump\OriginalLoopCompare\MP3_Normalized\Compress_exported3.mp3” NumChannels=“2”
TrackClose:
UnmuteAllTracks:
SelectNone:
LastTrack:
Toggle:
SelTrackStartToEnd:
TrackSolo:
Export2:Filename=“C:\Users\Admin\Music\AudacityEXPORTS\SingleLoopDump\\OriginalLoopCompare\WAV_Unedit\Unedit_exported2.wav” NumChannels=“2”
Normalize:ApplyGain=“1” PeakLevel="-1" RemoveDcOffset=“1” StereoIndependent=“0”
Export2:Filename=“C:\Users\Admin\Music\AudacityEXPORTS\SingleLoopDump\OriginalLoopCompare\WAV_Normalized\Normalized_exported2.wav” NumChannels=“2”
Export2:Filename=“C:\Users\Admin\Music\AudacityEXPORTS\SingleLoopDump\OriginalLoopCompare\MP3_Normalized\Compress_exported2.mp3” NumChannels=“2”
TrackClose:
UnmuteAllTracks:
SelectNone:
LastTrack:
Toggle:
SelTrackStartToEnd:
TrackSolo:
Export2:Filename=“C:\Users\Admin\Music\AudacityEXPORTS\SingleLoopDump\\OriginalLoopCompare\WAV_Unedit\Unedit_exported1.wav” NumChannels=“2”
Normalize:ApplyGain=“1” PeakLevel="-1" RemoveDcOffset=“1” StereoIndependent=“0”
Export2:Filename=“C:\Users\Admin\Music\AudacityEXPORTS\SingleLoopDump\OriginalLoopCompare\WAV_Normalized\Normalized_exported1.wav” NumChannels=“2”
Export2:Filename=“C:\Users\Admin\Music\AudacityEXPORTS\SingleLoopDump\OriginalLoopCompare\MP3_Normalized\Compress_exported1.mp3” NumChannels=“2”
TrackClose:
SelectNone:
LastTrack:
Toggle:
SelTrackStartToEnd:
SelEnd:
FirstTrack:
NextTrack:
NextTrack:
Toggle:
CursorRight:
SelPrevClip:
Split:
SelectNone:
LastTrack:
Toggle:
SelEnd:
Delete:
LastTrack:
TrackMoveTop:
TrackMoveDown:
TrackMoveDown:
TrackMoveDown:
Toggle:
FirstTrack:
NextTrack:
NextTrack:
SelEnd:
CursorRight:
Toggle:
SelPrevClip:
MoveToPrevLabel:
CursorLeft:
Toggle:
NextTrack:
NextTrack:
Toggle:
SelPrevClip:
SelPrevClip:
SelPrevClip:
SelPrevClip:
SelPrevClip:
SelPrevClip:
SelPrevClip:
SelPrevClip:
SelPrevClip:
SelPrevClip:
SelPrevClip:
SelPrevClip:
SelPrevClip:
SelPrevClip:
PrevTrack:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
Split:
Toggle:
SelNextClip:
Toggle:
SelectNone:
FirstTrack:
NextTrack:
NextTrack:
NextTrack:
Toggle:
SelEnd:
CursorRight:
SelPrevClip:
CursorShortJumpRight:
Toggle:
NextTrack:
Toggle:
SelPrevClip:
Silence:Use_Preset="
I DO A LARGE AMOUNT OF EXPORTS AT THE END NOT WANTING TO CONSIDER HOW MANY STEMS WERE MADE. THIS WILL THROW AN ERROR and A PROMPT AT THE END. IN AUTOHOTKEY I CHECK FOR THIS THROWN ERROR TO DETERMINE IT HAS FINISHED. THEN CLOSE OUT, RENAME the FOLDER THE EXPORTS WERE SENT TO BY THE NAME OF THE PROJECT AND RINSE AND REPEAT ON THE NEXT FILE IN A SPECIFIC FOLDER. …
Lastly sharing this helpful uitility but referenced macro as well. It helped with recording and AutoHotkey automation reading the screen so that labels are expected sizes.
XXX_Utility_Zooming.txt
Select:End=“2700” Mode=“Set” Start=“0” Track=“0” ZoomSel: SelectNone:
Thanks for joining me on this Audio Experience. Definately room for even more growth. But here I rest for the DK30 project. And in good time and company.
STAY SAFE AND SOUND
🤖RISE OF THE MACHINES🤖
! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! Project completed. ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !
Let’s observe in all it’s greatness. I was doing multiple things on the device at the time.
In conjunction with using AI to analyze audio, AI (ESRGAN) to intelligently scale (and unblur) images and Stable Diffusion. Not all three at once but in a combination to use one resource while not using another. Check out the ESRGAN and Stable Diffusion here: https://day9.tv/dk30/project/63446b83499cfd1e8e37fe39.
1hr 9min 55sec in 5 SECONDS
The process which was manual and took me 12 hours at start 5 months ago is now shortened to about 5 minutes per request …unattended [excluding some setup]. It’s never been automated beyond the macro hotkeys until now. Thanks for the DK30 challenge.
I see many opportunities to improve this automation. One, running it during a stream rather than after the record. Additionally it was a good exploration of controlling devices unattended.
What more fun can be!?
Some more stream version audio loops for our listening pleasure exported from running the above gif.
TWICE-CantStopMe
HollywoodUndead-S.C.A.V.A. (Piano Expansive)
TWICE-MORE&MORE
InkubusSukkubus-Wytches
ko-ko-ya-女心と秋の空(OnnagokoroToAkinosora)
Halo2Anniversary-MainTheme
Suikoden2-GothicNeclord
FinalFantasyX-2-Eternity-MemoryOfLightwaves
Reminder these are without post editing and hence called stream versions. I want to still remix each. Even going further to “encode” the remix from a text structure with beat finding understandings but that’ll be a different challenge. I bbelieve this challenge rests here.
With only some documentation adn summary to close out this DK30 project remaining…
ANd yet more automation use. This is so amazing… Why no one has tried this before…they have had to - right?
TheLegendOfZelda_GreatFairyFountain
ChainsawMan-OpeningFull[KICKBACK]byKenshiYonezu
PharrellWilliams-MarilynMonroe
RequiemForADreamSoundtrack
LiSAHomura『炎』
NationalAnthemLuxembourg-OnsHeemecht
Even though I have the ‘stems’ from each of these also ripped I’ve been providing how they sound on stream so I can focus on the automation more and more before reaching further into remixing again (as heard in the very beginning).
I’m discovering that the automation pauses at some key spots and think it has to do with active window messages and trying to force the input into Audactiy’s “window” while it is not the focus. I’m going to resolve some of that as things continue.
A cool use of these stream versions has been recapping the week’s music prowess and activity for voting, and some side happy-people-relations - providing the versions to the requesters on Twitch so they leave with something they can use rather than having to skim VODs which inevitably will be removed in time.
Some catching up to do. No worries - no loss of time off my clock except for some setup still. Otherwise it’s all automated.
TheAvalanches-FrontierPsvchiatrist
Skillet-NotGonnaDie (Piano Expansive)
TheLegendofZelda-MainTheme
GénérioueJoueurDuGrenier(YannickCrémer)
Castlevania-WanderinoGhosts
Stray-TheWayYouComputeTonight
If you’re liking what you’re hearing then consider stopping by and checking out Jonathan Ong on Twitch @ https://twitch.tv/jonathanong Or better yet pay for a request and I’ll be AUTOMATA-PEA-IT.
You may have also noticed that there’s a “Piano Expansive” - that request was one of mine. I’m very fond of the piano instrument. And it’s everything. Instead of doing the song as it’s played I enjoy creativeness and requested only a segment of it - the intro - be used and for the streamer to make everything else up from the material. Expand the composition without listening to the rest as if they were writing the song. All limitations and parties considered it’s quite attractive.
Final push. Tons of copi-paste splashing going to happen.
Looked into exploring what may already exist for a consumer on creating searchable voice logs and voice transcription software. As before I refound Google Translate as easy accessible and highly regarded. But it keeps cutting out.
I changed my search for accessibility/handicapped settings or fine-tuned it for closed captions and found the following resources.
Resources:
- Closed Captioning for the Blind: https://www.seeingwithsound.com/caption.htm
- Adding Closed Captions to a Video: https://www.youtube.com/watch?v=aSYWjLkL6To
- Veed.io: Closed Caption adding (sign-up based): https://www.veed.io/
- Downsub to download subtitles after created: https://downsub.com/
- Or simply in YouTube copy-paste after expanding the transcription.
- LALAL.AI )paid service): https://www.lalal.ai/guides/how-to-extract-voice-from-any-video-or-movie/
- Find Song Key (free): https://getsongkey.com/tools/key-finder
Of these VEED looks promising for longer VOD videos. And then Downsub to get it in a SRT file for easy searching and indexing.
These subtitles can likely be used to automate “clipping” and discussions around threads from streamers. Out of scope but neat.
New stream. New loops. Automatic Stream capture loops and Stems. I did not share the stems below. Only the automation on the stream versions.
TheMandalorian_MainTheme
Générioue_JoueurDuGrenier(YannickCrémer)
MarioKart64_KoopaTroopaBeach
Muse_Uno
EllieDixon_Swing!
Persona4_ReincarnationSpecialist
TheIdolm@sterLiveThe@terPerformance06_FindYourWind!
BillieEilish_KhalidLovely
The speed is great. The automation was consistent betweem all nine. There were a few issues only observed in the first startup.
Seems like the next step is working on indexing the stream so can do more detailed separation and loop finding. Will work on those areas next.
Will briefly to see if I can solve the problem by silencing segments I don’t want rather than finding those I do.
Also, still, the naming on exports/ folders I now want to match the name of the original .aup3 file with an appended number.
Added Audacity Prev/Next Clip Boundaries Error to the Audacity forum: https://forum.audacityteam.org/viewtopic.php?f=46&t=127670&p=458718&hilit=repeat#p458718
In learning about automating the tasks I’ve additionally come across the following resources.
Of these “AutoHotkey Search by Text Conversion for Images” and “Image Similarity 1” are possible uses to aid in thew automation tasks as they might be able to search the screen and determine progression of the stream for determining camera changes, requesting, and such. For label making.
The others are fun interesting concepts used in other areas…
Additional Resources:
- AutoHotkey Search by Text Conversion for Images: https://www.youtube.com/watch?v=aWRAtvJq9ZE
- AutoHotKey Enthusist built “Automate My Task” https://www.youtube.com/watch?v=0TkuIiUd6t0
- Image Similarity 1: https://www.youtube.com/watch?v=16s3Pi1InPU
- Image Similarity 2: https://pypi.org/project/image-similarity-measures/
- Microscopic similarities 1: https://www.apeer.com/home/
- Microscopic similarities 2: https://www.youtube.com/channel/UC34rW-HtPJulxr5wp2Xa04w
- Microscopic similarities 3: https://github.com/bnsreenu/python_for_microscopists
Here are some cool Audio Movement Tools:
- Hand Motion Controller Synth: https://www.auug.com/
- Gloves Movement Controller Synth: https://mimugloves.com/
More progress.
The main process is constantly being updated. I’ve added:
- Swapped the export to the end
- Made it compress using MP3 instead of OGG for Safari users
- Gets Rough ~10sec read on loop length. Assumes loops are no longer than 1minute50seconds but easily expandable.
- Uses reading loop length to better label
- Some testing in audio clip boundaries inside Audacity and unreliable results when repeating. Very strange - to be reported to Audacity forum soon.
- Opens all files inside a specific folder
- Renames the file folders based on the name of the original .aup3 file.
As well as played around with testing splitting the main loop. Turns out there is no fully accurate information, only close. However close is not constant and will lead to misleading separations. Inaccurate.
So now I’m improving the stream reading which brings in other concepts and AI.
Taking a step back, this is content modified from a streamer, Jonathan Ong (twitch.tv/jonathanong).
I can use the stream itself - or a recording of it to analyze when loops are performed. Let’s get started on analyzing the stream.
To start I added voice recording in AutoHotkey for the stream.
This can be improved. Ideally multiple processes in parallel will run to gather the camera changes, loop starting/ending, stream starting/ending, and whatnot so all streaming processes can be entirely automatic.
Here are the stream versions extracted along with the stems yet to be remixed.
Pokémon Rubv & Sapphire - Littleroot Town
Fraagle Rock - Down at Fraggle Rock {2022)
Ongriginal - D Flat Major
Angel Beats - MvSoul
Hoshino Marine - Unison
Tales from the CQ’Dt - Theme
Elder Scrolls Oblivion {Jeremy Soule) - Reign of the Septims
Final Fantasy IX - Terra
Dream Theater — Stream of Consciousness3
Darren Hayes - Insatiable
HarlemYu-QingFeiDeYi情非得已
PinkFloyd-Money
Not done yet!
I’d like this to be even closer to fully-automatic
I’d ideally want:
- Camera Stage Direction Detection
- Voice Recording
- AutoBackup of Stream
- Separation of Loops
- Discovery of Piano Arrangements
- Autolabeling sound tracks from audio
- Linking conversation threads
- Annotations from Grand Piano for Midi data
- Detection of loop segment accurately
- Requesting songs
- Posting Songs
- Automatic Chatting / Responding to chats / Events
Turns out one thing leads to another and with accurate stream consistency then automating the naming will become much easier.
As you can see there are many ways to use the automation for a streamer. I want to focus on theose areas (not in any order) next.
This project is coming along but it’s result is not as flashy. Documentation may be a Google Folder as well as posted here individually - that’d be quite a bit of “garbarge” looking code to sleuth through.
Since this is the thrid week for the DK30 and projects are comign to a close. It is likely outside the scope of actually using a program and interpretting lines to tell Audacity how to rebuild the loop and remix it automatically. That will be outside this DK30 project.
This is so nasty! 50x speed. 15 minutes, 5 loop builds completed. Folders made, more exports…categorized.
So the AutoHotkey script is now able to work across files. You can see this everytime it goes to a black screen.
Meaning the loops can be setup. Mind blowing capabilities. Iin practice this is a savings of ~30 minutes more.
I’ve even added the export of the “short EASY-segment” at the start. Though that causes a timing confusion - so I may move it around a bit.
I’ve updated it so that it is rarely operating on delays but have some to buffer around the edges. Small stress test - as I was using thec omputer in other regards while this process ran and was recorded (though improvements there still desired).
Realistically I watched it and it had two hiccups due to the stress test, delays, and overall function. Fixable with more work.
More to come:
- Improvements in the readability of the AutoHotkey script
- Better Layer & Overall Song Naming
- Automating more processes so I can further allow the separation of the loops, isolating the vocals to be included in the automated process!!!
Little by little.
I’ve got new loop material.
I won’t have sufficient time over the next 48 hours to actively work on this unfortunately. But I did some preliminary checks on how I might go about automatically stripping the audio from the large looper.
As you don’t know because I’ve been not explaining things well, the loops I’m stripping are part of a larger audio stream. The audio stream has multiple loops in it, but is just an audio stream. I’m desiring a um way to automatically grab those loops from the audio stream.
Welcome to amplification and labeling.
Here I’m showing that I amplified the audio stream by +24dB and intentionally caused the peaking (red areas). The audio is zoomed out far enough in Audacity that this shows as constant red however if I were to zoom in, there would be areas of no peaking in the red sections.
You can see some areas where the loop starts because of low volume that are not peaking.
Then you can notice my attempts (these were the closest) in tweaking the parameters of labeling the sound to find the segments of the loops alone.
This still needs more work but if I can get it to operate, I will save time and have automated another step in the process further.
I will try to address these next:
- Loop finding from the audio stream
- Short-EASY-segment
- Naming (the most challenging and uninteresting but most important)
Hope to have more to show after 48 hours. Here here to week 2!
The main process is now fully automated and we’re moving toward automating the additional ones.
This process:
- Creates a new folder to store the exports (seen in the bottom-left)
- Labels the sounds
- Iterates N times based on the number of labels
- Extracts the stems
- Exports the Stems to the made folders as WAV (unedit), WAV (normalized), and OGG (normalized, compressed)
I want to go further and improve so that this process is not reliant on a delay but realises when to run the next iteration. You can see the delays since the automation process aligns across multiple here…
That feels good to watch the automation at work. No, each of these is done individually - I simply edited them together to show the delays match and using it across multiple.
By bringing this to scale I noticed more areas to be worked on. Majorly the naming has to be tweaked so that I use the correct labels. Labeling is very important. From doing so many at scale I got myself spun around as to which loop is which. Willy Wonky.
I’m also working on the outside-the-main-process to automate splitting the individual loops from the looper without attended action.
So I’m going to rerecord the process with more helpful information on screen and present a picture of that soon.
In general the um process is as follows: Gather the looper audio and split at the individual loops. I’m using Audacity with the FFMPG plugin to do this. It looks something like this.
Then I manually find the looped segment to start. To do this I’m using “peak finder.ny” a nyquist plugin. I find two peaks in the audio and then split the audio and check it on those peaks, inverted on the original audio. This also tells us the loop length. It’s quite simple to do.
Afterwards we performs a series of similar steps of repeating and inverting the loop segments from labels generated at points when talking occurs. Until the whole loop is iterated.
It then cleans up all the segments and exports each into a folder. Three exports happen:
- 1 >> The WAV / “unedit” is the original audio after these steps.
- 2 >> The WAV / “normalize” is the audio normalized and usually the stems I use in remixing
- 3 >> The OGG / “compressed” is the audio compressed to the OGG format.
Recently I learned that OGG does not read well on Safari. I could test that but instead I will likely flip to using the MP3 compression method.
I’ll get a better econdary recordig of this posted soon here.
So I had this idea that I could put the material together on a website. It’s out of scope for the project. But I’m linking some resources I got from asking a friend for details on using “audio” online.
Resource to look at for manipulating audio in a website:
- A fun example: https://jamesfiltness.github.io/web-audio-mixer/
- Key of the music and Beat/Tempo: https://vocalremover.org/key-bpm-finder
- Web Mixer Example #2: https://audiomixer.io/
- HTML Audio element: https://www.w3schools.com/html/html5_audio.asp
- Useful ‘working’ projects to select as a baseline or try-and-fail-to-learn: https://www.npmjs.com/search?q=audio%20mixer
I figured out images, I’m hopeful I can post audio embedded on this page as well. I’ve been focused on trying something with combining three separate covers of the same song into one.
The three songs are: Orchestral (Brass focus) Loop:
Vocal focused Loop:
and Synth focused Loop:
These I stripped the stems for the layers manually with the help of automated-hotkeys…that is “macros” in Audacity.
Once I had the layers/stems from the audio loop I rinsed and repeated for the other two. Once I had all layers/stems I added it to Fruity Loop (DAW) and mixed it together.
The following is 41 selected layers from the 62. I wanted to show the general results from a mild manual ripping.
Here is the altogether REMIXed piece from the above three loops.
I realise I didn’t explain how it works fully - or at all really. I’ll explain as I go but that will leave things with some strangeness at times.
Things I noticed from doing this practice:
- Tails for music from the looper will be challenging
- Possible to carry music through styles
- Need to learn more about time stretching for matching iterations. Humans are not tempo perfect and introduce changes in melody
- Want to learn more about music Key
- Vocals don’t always work and can be difficultr to add to an instrument heavy piece without lyrics
- Distortion might occur. Will want to look at simple ways to adjust frequencies to remove that. Equalizers are the resolution not the tool I think.
- Different devices might reveal the distortion differently. I listened to this on different speakers and felt more immersed from one over the other.
- “Stings” or short solo/fanfares are helpful in moving the interest of the piece
- Mixer should reduce volume not increase but for some reason couldn’t use that rule in spots and decided to increase the gain ffrom the mixer end.
- Other things to be determined as I listen more and more.
Here is an example of a differeent loop but with OGG files of each individual layer for your listening pleasure. The link will likely not stay active forever: https://drive.google.com/drive/folders/1TW55QusJ7Jz3K-oBX3UF7mioYQ6ALpiS?usp=sharing.
From that link you can hear the ‘altogether’ but also the individual layers and how “bad” or “good” they are. This example isn’t showing in what order the layers were extracted. Typically the worse layer is last by nature of how I’m performing the rip iteratively but that is not always the case.
Chaos in the looper which I think has something to do with “loudness” or a “Frequency” or “reverb effect using the on-board CPU load” will cause the looper to stagger and offset the rip creating inaccuracies.
A breif secondary update:
This was a third start to the project using Audacity MACROS and NYQUIST (LISP interpreter): https://forum.audacityteam.org/viewtopic.php?f=46&t=126543&start=10
Here is more information and a very helpful guide to using Nyquist in Audacity. https://audionyq.com/professional-sounding-fade-out/
I realise this updates is not discussion about the actual automation program. Once I have played around with this sample “short hand” version I’m making from stems the updates will look more closely at iterating on the program.
This weeks focus has been getting a sample of the Mixed “short hand” track to help show a result. I plan to share some of those by the end of the week and compare with previous.
Time has been on having three versions of the same track performed differently. +One track performed with brass and orchestra +One track performed with vocals and singing +One track performed with synths and “Stings”
I’ve tried taking the strings from another track which is considered a different song altogether but has similar overlaps. I’m fairly certain I can get that to work but my first obstacle became “Time Stretching”.
I’m working on implementing time stretching on the individual stems of the tracks so that they match up and are all played “in sync”/“in time” with each other.
Here is the FL Studio reference: https://www.youtube.com/watch?v=mEA9AlKXXD0
So I tested this out at the start of the week. It did not go as I desired. Perfect! There are some rooms for improvement. Fortunately I um have a manual override for stem ripping which takes about 10-15 minutes and in some ways faster than I plan this automation to be because I’ll intentionally add waits in places to allow for processing.
I’m looking at how I can share music on this project after just learning about images. I’ll likely use YouTube since I already made an account and it’ll provide a secondary ability to give the music a video representation using the other DK30 project with AI artwork.
I came across BandCamp and Music Gateway. Investigation necesary on how Music Gateway - which I’ve not heard of - is a more suitable application in this case.
Once I get that setup further I’ll be able to share the audio in abetter way.
Estimated Timeframe
Oct 19th - Nov 19th
Week 1 Goal
Show normal results from the program run manually. Show loop separation quality Show mixed down “short hand” loop.
Week 2 Goal
Record running automated.
Week 3 Goal
Closer attempt to run fully-automatic.
Week 4 Goal
Documentation and package files for use.
