NiFi - Lesson 14 - NiFi SQL Query Record Processor

Опубликовано: 30 Сентябрь 2024
на канале: Vikas Jha

5,424

connect on Fiverr for job support: https://www.fiverr.com/automateanythin
.
. For meeting: https://calendly.com/learnwithvikasjh...
.
. for more free & premium courses: https://itpanther.com/
.
. Learn More Here: https://www.itpanther.com ubscribe to upport the channel: https://youtube.com/c/vikasjha001?sub...
Need help? Message me on LinkedIn:   / vikasjha001
Want to connect on Instagram? Here is my id @vikasjha001 Connect to me:
💥 LinkedIn
  / vikas-kumar-jha-739639121

📷 Instagram
  / vikasjha001

✈️ Channel
   / lifebeyondwork001

Apache NiFi supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic. ome of the high-level capabilities and objectives of Apache NiFi include:

Web-based user interface
eamless experience between design, control, feedback, and monitoring
Highly configurable
Loss tolerant vs guaranteed delivery
Low latency vs high throughput
Dynamic prioritization
Flow can be modified at runtime
Back pressure
Data Provenance
Track dataflow from beginning to end
Designed for extension
Build your own processors and more
Enables rapid development and effective testing
ecure
L, H, HTTP, encrypted content, etc...
Multi-tenant authorization and internal authorization/policy management

QueryRecord
Description:
Evaluates one or more QL queries against the contents of a FlowFile. The result of the QL query then becomes the content of the output FlowFile. This can be used, for example, for field-specific filtering, transformation, and row-level filtering. Columns can be renamed, simple calculations and aggregations performed, etc. The Processor is configured with a Record Reader Controller ervice and a Record Writer service so as to allow flexibility in incoming and outgoing data formats. The Processor must be configured with at least one user-defined property. The name of the Property is the Relationship to route data to, and the value of the Property is a QL ELECT statement that is used to specify how input data should be transformed/filtered. The QL statement must be valid ANI QL and is powered by Apache Calcite. If the transformation fails, the original FlowFile is routed to the 'failure' relationship. Otherwise, the data selected will be routed to the associated relationship. If the Record Writer chooses to inherit the schema from the Record, it is important to note that the schema that is inherited will be from the Resultet, rather than the input Record. This allows a single instance of the QueryRecord processor to have multiple queries, each of which returns a different set of columns and aggregations. As a result, though, the schema that is derived will have no schema name, so it is important that the configured Record Writer not attempt to write the chema Name as an attribute if inheriting the chema from the Record. ee the Processor Usage documentation for more information. Buy my Udemy courses at a special price for a limited time:
✅ https://www.itpanther.com/prometheus-...
✅ https://www.itpanther.com/grafana
✅ https://www.itpanther.com/python
✅ https://www.itpanther.com/liferay
✅ https://www.itpanther.com/apachenifia...
✅ https://www.itpanther.com/apachenifi
✅ https://www.itpanther.com/redis

🤝 For collaboration or other inquiries connect
📧 Email: [email protected]
📞 Whatsapp: +917042103414