LONG- BUT HOPEFULLY INTERESTING- STUDY DONE ON HS TEOH'S THREAD FOR FUGUE IN G, WHERE OPINIONS LEANED TOWARDS THE POLINA RECORDING
(I would have posted this in the original thread for consistency, however, it seems you cannot attach images in replies to threads- only when creating them.)
Against my better judgement, I allowed myself to become a bit obsessed with the phenomena in that thread.
Firstly, I want to make it clear that my intention is not to prove anyone "wrong" for favoring one rendition of the piece over another. Musical preferences are subjective, and each person is entitled to their own taste. I'd like to emphasize that this isn't my composition; my personal stake in this matter is minimal, but certain aspects of the discourse surrounding Polina's version bothered and intrigued me. Through this study, I aim to raise awareness among fellow composers to be wary of situations and products like this. Blindly supporting them and acknowledging their authenticity, in my opinion, is detrimental to the very field in which we on this forum are working in.
What I hope to accomplish through the study provided below is to prove two main cases:
1. Polina's recording is a MIDI-produced rendition, lacking the authenticity of a live performance
.
2. The belief, fostered prior to hearing either version, that Polina's recording was a genuine human performance led participants in this thread, consciously or subconsciously, to favor her version over ours, considering they were aware that HS Teoh and I's version was computer-generated.
I acknowledge there may be other factors influencing decisions, such as a desire to support HS Teoh's effort in recording the piece, giving a sense of confirmation to Polina's paid, hired endeavor. Additionally, some may have been put off by my statement declaring, "I am quite certain that with the MIDI data for your Fugue here, I could provide a result that is still 100% inauthentic but perhaps quite a bit more convincing." While admittedly a bit condescending, I suspect this statement may have factored into people's preferences - but please read on and consider my thoughts.
To begin, I wanted to collect an unbiased sample set without any preconceptions of either recording. I presented both versions to a music review site under the "Classical" category, allowing 20 random users interested in classical music to rate the pieces based on predefined categories (HS Teoh was duly credited for the composition, with a link to his Soundcloud channel provided) Users were asked to rate the versions on the following criteria:
⦁ Expression of performance
⦁ Quality of recording
⦁ MIDI or played by a human being?
In terms of the 'weighted average' (which considers whether ratings were above or below each person's average rating- it is not related to the 3 individual categories, but rather, the user's "overall" rating for the piece), HS Teoh+David's Version scored significantly better than Polina's version- by 69 points, or 363.16%. More than half the raters rated Polina's version below average, while on the other hand, more than half the raters rated HS Teoh+David's Version above average.
The results of the two versions in each category can be seen below;
Polina's Version -
HS Teoh+David's Version-
Observing and summarizing the comments, users did not seem entirely convinced on either version in terms of whether it was a human or MIDI. However, some users who heard and rated both versions strongly felt that the HS Teoh+David version was more likely to be a human or superior. On the Polina version, there were several comments regarding robotic playing and a MIDI feel. For the HS Teoh+David version, some users were genuinely unsure. However, many noted a lot of expression in dynamics and playing in this version, with a lack of 'robotic' comments.
Here is an AI analysis that anaylzed and compared solely the text feedback between each version: (Beneath this summary is the feedback provided by every user for each verison for you to read)
General Observations:
Version 2 (HS TEOH+David) is praised for its emotionality, dynamics, and expressiveness.
Version 2 (HS TEOH+David) is perceived as having a more realistic human touch, but there is some uncertainty.
Version 1 (Polina) is acknowledged for its technical proficiency but criticized for sounding too mechanical and MIDI-like.
Sound quality and mix are concerns for Version 1 (Polina).
Conclusion:
Based on the feedback, it seems that Version 2 (HS TEOH+David) is generally praised for its expressiveness, dynamics, and authenticity, with some listeners leaning towards it being a human performance. On the other hand, Version 1 (Polina) receives positive comments on technique but faces criticism for sounding robotic or MIDI-like, with a mention of a poor mix.
Therefore, based on the provided comments, Version 2 is perceived as superior by the audience for its authenticity, expressiveness, and overall musicality.
(You can click each image below to enlarge and enhance for reading clarity)
While the previous feedback was valuable, I sought the perspectives of individuals with expertise in music production, particularly those experienced in MIDI and virtual instruments. Using the same 'blind taste test' approach, without any preconceptions indicated, I started a thread on a prominent website catering to this demographic in the "MIDI" sub forum- and that yielded results that, though limited, once again leaned towards favoring version 2 (HS TEOH+David). Some respondents were firmly convinced that Polina's version relied on MIDI, with HS Teoh and I's version considered authentic. Others were less certain, but notably, only one comment acknowledged maybe detecting any human elements in Polina's rendition. It's crucial to note that many contributors on the site actively work with MIDI data in professional roles, yet it is important to remember this is still, to some capacity, a subjective matter. You can find a copy of that thread attached here.
And finally, in my most direct attempt to conclusively determine the authenticity of Polina's recording, I reached out to her directly on Fiverr (a website for hiring freelance musicians). I offered to cover the expenses for a recording and video of HS Teoh's fugue. Unsurprisingly, she declined, citing the "closure of the conservatory for winter due to heating issues caused by shelling". Additionally, she mentioned a "heavy workload related to concerts and sessions."
In response, I assured her that I would find a nearby studio with a piano and arrange for a videographer. However, my offer was met with stony silence, and I received no further replies despite various persuasive attempts. I had a friend reach out after a few days on an unrelated matter - he received an answer within 30 minutes.
Gearspace - Is this performance MIDI or played by a human_.pdf
Now, let's consider the logic of this entire situation: if I were a pianist who had presumably already learned and recorded a piece as technically correct as we hear it in Polina's recording, this would be an 'easy money' opportunity and a chance to showcase my talent professionally.
I would be sure most pianists, confident in their skills and aspiring to develop their careers, would likely welcome the opportunity to record in a studio and provide a performance video of a difficult, technical, and brilliant piece. Such content could significantly enhance one's online presence and attract more opportunities. The refusal to accept the offer raises questions for me personally, suggesting a potential fear that a studio recording might reveal an inability to actually play the piece. Various excuses, such as being too busy or lacking access to a suitable piano, were addressed in my offer.
If we give her the benefit of the doubt regarding her busy schedule, it still doesn't align logically. Assuming she presumably learned and played the piece rather recently, it would only take a couple of hours to go into the studio, record it, and be done with it. The piece is short (less than 3 minutes) and even in only an hour session, many takes could be had.
Let's also consider the turnaround time and overall logic of the services Polina offers on Fiverr. A 'Premium Recording,' which includes up to 300 seconds of music recorded with four revisions within a 5-day delivery time, is priced at $40. While I don't know the specific details on the arrangement made with HS Teoh before the recording (perhaps she charges more or prices out more difficult pieces differently) I am assuming Polina operates based on the services listed on the site; and it would seem impractical and unusual to learn a piece of this technical demand for only $40.
Why?
Well, even for an accomplished and virtuoso pianist, it would take at least 8 hours to sort out all the fingerings and decide on the interpretation — and that's only half the battle. This is a new piece. There isn't a glossary or history of scores marked up from various artists prior from which to pull. Figuring out the proper phrasing of each voice is even more challenging. If you're really good, and have played lots of fugal music, most of the material presented might not be quite as challenging- but when coupled with learning the piece and bringing it up to tempo, it doesn't make sense to work at such a rate,which would be an incredibly low rate per hour given the time required to learn this piece. While it's not the most technically demanding fugue, it is tough. An advanced pianist may be able to make substantial progress in a short period, but to me it seems producing a clear and very coherent recording in just 5 days is still an ambitious goal. Now, the caveat for this is the fact that I don't really know how long Polina apparently spent learning this work.
What does seem to make sense to me, however, is that one could work at this rate if they were scanning sheet music, converting it to MIDI (or perhaps she requested the MIDI data, I am not sure), and then editing the MIDI data to attempt to make it convincing using a digital piano and software. This can be done in less than 8 hours, making it financially viable compared to the time spent.
So finally, after considering all of this, let us question why all members here (except for Mike Hewer and myself) tended to side with Polina's version when, based on unbiased sample sets with less context, it seems that the HS Teoh + David version is preferable outside of this forum. It could simply just be a small sample size in terms of the members who participated in this thread. But most likely, I point to the conclusion of confirmation bias. I draw a comparison to one well-known study—the "Price-Quality Relationship in Wine," conducted by Frederic Brochet, a researcher in enology.
In one part of Brochet's study, participants were given two glasses of wine—one labeled as an expensive wine and the other as a cheap wine. However, both glasses contained the same wine. The participants consistently rated the supposedly "expensive" wine higher in terms of taste, aroma, and overall quality. The participants' perceptions were influenced by the expectation that the higher-priced wine would be of better quality.
That study, and the mini study I've done here, highlight the impact of psychological factors—in this case, the preconception of a work being performed by a human versus a computer. People's beliefs about the quality of wine can be influenced by factors beyond the actual sensory characteristics of the wine itself—so why not a musical recording?
Replies
Your insistence piqued my curiosity enough to start a little sleuthing of my own. Probably also against my better judgment, but the result was interesting, to say the least.
I had recently just finished an order with another Fiverr pianist (whom I do not wish to name at present for certain reasons), also with a complex fugue. In the course of the order I had requested certain revisions, and I believe beyond doubt that the corrections were made by either editing the MIDI or stitching it into place after recording it separately. Which, incidentally, I have no problem with, given the complexity and length of the piece, and also the price I paid for it -- I would hardly have expected her to retake the entire thing.
Anyway, since this discussion just came up at around the same time, and this particular pianist does in fact offer to record videos of her performances unlike Polina (albeit only with prior agreement), I thought it was a good opportunity to probe a little about her willingness to record a video of the piece that she has ostensibly just learned. So I casually asked her how much she would charge to record a video of the piece -- not intending to actually go through with it, but just to gauge her willingness to do it, saying that I wanted to know only for future planning purposes, as I currently don't have the budget for it. I was expecting to be quoted an exorbitant price, given the difficulty of the piece, you see.
Her answer, however, was quite revealing: she said that it was a very difficult order and that she's not sure if she could take it, explaining that doing an audio was possible because she could record in sections, but she was not sure whether she could play through the entire piece on video (I guess that would require her to do it in one take). I thought it is very interesting that she would not even consider the possibility, even though she could have quoted a high price (e.g. to pay her expenses while dedicating her time to learn the piece to the level she could do a video), and/or set a long deadline to allow herself enough time to pull it off. Since I said that I wanted a quote for planning purposes, implying the possibility that I would be willing to plan for actually funding her efforts to this end, she could have quoted me some exorbitant price that, should I actually take it up, would represent a huge step in her career as a pianist in adding a very complex fugue to her portfolio, and being funded while doing so. The fact that she outright turned down the possibility is very interesting indeed, and while she isn't Polina, I would imagine that what this says about her work wouldn't be too far off from representing Polina's as well.
-
1
-
2
of 2 Next