Google Dataflow, How To Wait For External Webhook When Transforming A Collection?

I have a code that reads an Xlsx file, and for each line, do a process on a specific column. The problem is related to the 'Transform' part of the Dataflow. I implemented a specifi

Solution 1:

Not sure if I fully understood your question but seems like you are asking if Beam offers a way for a DoFn.process() method to be called once a given callback has been invoked. Currently Beam does not offer such a feature.

What you can do here is to wait within the ComputeWordLengthFn.process() method till the request for a particular element has been completed (Exact way to perform that wait depends on the external API).

Please let me know if I misunderstood your question.