This project has retired. For details please refer to its Attic page.
MesosSchedulerDriver
JavaScript is disabled on your browser.
Skip navigation links
org.apache.mesos

Class MesosSchedulerDriver

  • All Implemented Interfaces:
    SchedulerDriver 

    public class MesosSchedulerDriver
extends java.lang.Object
implements SchedulerDriver
    Concrete implementation of a SchedulerDriver that connects a Scheduler with a Mesos master. The MesosSchedulerDriver is thread-safe.

    Note that scheduler failover is supported in Mesos. After a scheduler is registered with Mesos it may failover (to a new process on the same machine or across multiple machines) by creating a new driver with the ID given to it in Scheduler.registered(org.apache.mesos.SchedulerDriver, org.apache.mesos.Protos.FrameworkID, org.apache.mesos.Protos.MasterInfo).

    The driver is responsible for invoking the Scheduler callbacks as it communicates with the Mesos master.

    Note that blocking on the MesosSchedulerDriver (e.g., via join()) doesn't affect the scheduler callbacks in anyway because they are handled by a different thread.

    Note that the driver uses GLOG to do its own logging. GLOG flags can be set via environment variables, prefixing the flag name with "GLOG_", e.g., "GLOG_v=1". For Mesos specific logging flags see src/logging/flags.hpp. Mesos flags can also be set via environment variables, prefixing the flag name with "MESOS_", e.g., "MESOS_QUIET=1".

    See src/examples/java/TestFramework.java for an example of using the MesosSchedulerDriver.

    • Constructor Detail

      • MesosSchedulerDriver

        public MesosSchedulerDriver(Scheduler scheduler,
 Protos.FrameworkInfo framework,
 java.lang.String master)
        Creates a new driver for the specified scheduler. The master should be one of: 
         
 host:port
 zk://host1:port1,host2:port2,.../path
 zk://username:password@host1:port1,host2:port2,.../path
 file:///path/to/file (where file contains one of the above)

        The driver will attempt to "failover" if the specified FrameworkInfo includes a valid FrameworkID.

        Any Mesos configuration options are read from environment variables, as well as any configuration files found through the environment variables.

        Parameters:
        scheduler - The scheduler implementation which callbacks are invoked upon scheduler events.
        framework - The frameworkInfo describing the current framework.
        master - The address to the currently active Mesos master.

      • MesosSchedulerDriver

        public MesosSchedulerDriver(Scheduler scheduler,
 Protos.FrameworkInfo framework,
 java.lang.String master,
 Protos.Credential credential)
        Same as the other constructors, except that it accepts the newly introduced 'credential' parameter.
        Parameters:
        scheduler - The scheduler implementation which callbacks are invoked upon scheduler events.
        framework - The frameworkInfo describing the current framework.
        master - The address to the currently active Mesos master.
        credential - The credentials that will be used used to authenticate calls from this scheduler.

      • MesosSchedulerDriver

        public MesosSchedulerDriver(Scheduler scheduler,
 Protos.FrameworkInfo framework,
 java.lang.String master,
 boolean implicitAcknowledgements)
        Same as the other constructors, except that it accepts the newly introduced 'implicitAcknowledgements' parameter.
        Parameters:
        scheduler - The scheduler implementation which callbacks are invoked upon scheduler events.
        framework - The frameworkInfo describing the current framework.
        master - The address to the currently active Mesos master.
        implicitAcknowledgements - Whether the driver should send acknowledgements on behalf of the scheduler. Setting this to false allows schedulers to perform their own acknowledgements, which enables asynchronous / batch processing of status updates.

      • MesosSchedulerDriver

        public MesosSchedulerDriver(Scheduler scheduler,
 Protos.FrameworkInfo framework,
 java.lang.String master,
 boolean implicitAcknowledgements,
 Protos.Credential credential)
        Same as the other constructors, except that it accepts the newly introduced 'implicitAcknowledgements' and 'credentials' parameters.
        Parameters:
        scheduler - The scheduler implementation which callbacks are invoked upon scheduler events.
        framework - The frameworkInfo describing the current framework.
        master - The address to the currently active Mesos master.
        implicitAcknowledgements - Whether the driver should send acknowledgements on behalf of the scheduler. Setting this to false allows schedulers to perform their own acknowledgements, which enables asynchronous / batch processing of status updates.
        credential - The credentials that will be used used to authenticate calls from this scheduler.

      • MesosSchedulerDriver

        public MesosSchedulerDriver(Scheduler scheduler,
 Protos.FrameworkInfo framework,
 java.util.Collection<java.lang.String> suppressedRoles,
 java.lang.String master,
 boolean implicitAcknowledgements)
        Same as the other constructors, except that it accepts the newly introduced 'suppressedRoles' parameter.
        Parameters:
        scheduler - The scheduler implementation which callbacks are invoked upon scheduler events.
        framework - The frameworkInfo describing the current framework.
        suppressedRoles - The collection of initially suppressed roles.
        master - The address to the currently active Mesos master.
        implicitAcknowledgements - Whether the driver should send acknowledgements on behalf of the scheduler. Setting this to false allows schedulers to perform their own acknowledgements, which enables asynchronous / batch processing of status updates.

      • MesosSchedulerDriver

        public MesosSchedulerDriver(Scheduler scheduler,
 Protos.FrameworkInfo framework,
 java.util.Collection<java.lang.String> suppressedRoles,
 java.lang.String master,
 boolean implicitAcknowledgements,
 Protos.Credential credential)
        Same as the other constructors, except that it accepts the newly introduced 'suppressedRoles' parameter.
        Parameters:
        scheduler - The scheduler implementation which callbacks are invoked upon scheduler events.
        framework - The frameworkInfo describing the current framework.
        suppressedRoles - The collection of initially suppressed roles.
        master - The address to the currently active Mesos master.
        implicitAcknowledgements - Whether the driver should send acknowledgements on behalf of the scheduler. Setting this to false allows schedulers to perform their own acknowledgements, which enables asynchronous / batch processing of status updates.
        credential - The credentials that will be used used to authenticate calls from this scheduler.

    • Method Detail

      • stop

        public Protos.Status stop(boolean failover)
        Description copied from interface: SchedulerDriver
        Stops the scheduler driver. If the 'failover' flag is set to false then it is expected that this framework will never reconnect to Mesos. So Mesos will unregister the framework and shutdown all its tasks and executors. If 'failover' is true, all executors and tasks will remain running (for some framework specific failover timeout) allowing the scheduler to reconnect (possibly in the same process, or from a different process, for example, on a different machine).
        Specified by:
        stop in interface SchedulerDriver
        Parameters:
        failover - Whether framework failover is expected.
        Returns:
        The state of the driver after the call.
        See Also:
        Protos.Status

      • stop

        public Protos.Status stop()
        Description copied from interface: SchedulerDriver
        Stops the scheduler driver assuming no failover. This will cause Mesos to unregister the framework and shutdown all its tasks and executors. Please see SchedulerDriver.stop(boolean) for more details.
        Specified by:
        stop in interface SchedulerDriver
        Returns:
        The state of the driver after the call.

      • abort

        public Protos.Status abort()
        Description copied from interface: SchedulerDriver
        Aborts the driver so that no more callbacks can be made to the scheduler. The semantics of abort and stop have deliberately been separated so that code can detect an aborted driver (i.e., via the return status of SchedulerDriver.join(), see below), and instantiate and start another driver if desired (from within the same process).
        Specified by:
        abort in interface SchedulerDriver
        Returns:
        The state of the driver after the call.

      • join

        public Protos.Status join()
        Description copied from interface: SchedulerDriver
        Waits for the driver to be stopped or aborted, possibly blocking the current thread indefinitely. The return status of this function can be used to determine if the driver was aborted (see mesos.proto for a description of Status).
        Specified by:
        join in interface SchedulerDriver
        Returns:
        The state of the driver after the call.

      • run

        public Protos.Status run()
        Description copied from interface: SchedulerDriver
        Starts and immediately joins (i.e., blocks on) the driver.
        Specified by:
        run in interface SchedulerDriver
        Returns:
        The state of the driver after the call.

      • launchTasks

        public Protos.Status launchTasks(java.util.Collection<Protos.OfferID> offerIds,
 java.util.Collection<Protos.TaskInfo> tasks)
        Description copied from interface: SchedulerDriver
        Launches the given set of tasks. See above for details. Note that this may add a default filter (see mesos.proto) for the remaining resources. Notably the MesosSchedulerDriver does so.
        Specified by:
        launchTasks in interface SchedulerDriver
        Parameters:
        offerIds - The collection of offer IDs.
        tasks - The collection of tasks to be launched.
        Returns:
        The state of the driver after the call.

      • launchTasks

        public Protos.Status launchTasks(java.util.Collection<Protos.OfferID> offerIds,
 java.util.Collection<Protos.TaskInfo> tasks,
 Protos.Filters filters)
        Description copied from interface: SchedulerDriver
        Launches the given set of tasks. Any remaining resources (i.e., those that are not used by the launched tasks or their executors) will be considered declined. Note that this includes resources used by tasks that the framework attempted to launch but failed (with TASK_ERROR) due to a malformed task description. The specified filters are applied on all unused resources (see mesos.proto for a description of Filters). Available resources are aggregated when multiple offers are provided. Note that all offers must belong to the same slave. Invoking this function with an empty collection of tasks declines offers in their entirety (see SchedulerDriver.declineOffer(org.apache.mesos.Protos.OfferID, org.apache.mesos.Protos.Filters)).
        Specified by:
        launchTasks in interface SchedulerDriver
        Parameters:
        offerIds - The collection of offer IDs.
        tasks - The collection of tasks to be launched.
        filters - The filters to set for any remaining resources.
        Returns:
        The state of the driver after the call.
        See Also:
        Protos.OfferID, Protos.TaskInfo, Protos.Filters, Protos.Status

      • killTask

        public Protos.Status killTask(Protos.TaskID taskId)
        Description copied from interface: SchedulerDriver
        Kills the specified task. Note that attempting to kill a task is currently not reliable. If, for example, a scheduler fails over while it was attempting to kill a task it will need to retry in the future Likewise, if unregistered / disconnected, the request will be dropped (these semantics may be changed in the future).
        Specified by:
        killTask in interface SchedulerDriver
        Parameters:
        taskId - The ID of the task to be killed.
        Returns:
        The state of the driver after the call.

      • acceptOffers

        public Protos.Status acceptOffers(java.util.Collection<Protos.OfferID> offerIds,
 java.util.Collection<Protos.Offer.Operation> operations,
 Protos.Filters filters)
        Description copied from interface: SchedulerDriver
        Accepts the given offers and performs a sequence of operations on those accepted offers. See Offer.Operation in mesos.proto for the set of available operations. Any remaining resources (i.e., those that are not used by the launched tasks or their executors) will be considered declined. Note that this includes resources used by tasks that the framework attempted to launch but failed (with TASK_ERROR) due to a malformed task description. The specified filters are applied on all unused resources (see mesos.proto for a description of Filters). Available resources are aggregated when multiple offers are provided. Note that all offers must belong to the same slave.
        Specified by:
        acceptOffers in interface SchedulerDriver
        Parameters:
        offerIds - The collection of offer IDs.
        operations - The collection of offer operations to perform.
        filters - The filters to set for any remaining resources.
        Returns:
        The state of the driver after the call.
        See Also:
        Protos.OfferID, Offer.Operation, Protos.Filters, Protos.Status

      • reviveOffers

        public Protos.Status reviveOffers()
        Description copied from interface: SchedulerDriver
        Removes all filters previously set by the framework (via launchTasks() or declineOffer()) and clears the set of suppressed roles. NOTE: If the framework is not connected to the master, the set of suppressed roles stored by the driver will be cleared, and an up-to-date set of suppressed roles will be sent to the master during re-registration.
        Specified by:
        reviveOffers in interface SchedulerDriver
        Returns:
        The state of the driver after the call.
        See Also:
        Protos.Status

      • reviveOffers

        public Protos.Status reviveOffers(java.util.Collection<java.lang.String> roles)
        Description copied from interface: SchedulerDriver
        Removes filters for the specified roles and removes these roles from the suppressed set. If the framework is not connected to the master, an up-to-date set of suppressed roles will be sent to the master during re-registration.
        Specified by:
        reviveOffers in interface SchedulerDriver
        Parameters:
        roles - The collection of the framework roles to be revivied. If empty, this method does nothing.
        Returns:
        The state of the driver after the call.
        See Also:
        Protos.Status

      • suppressOffers

        public Protos.Status suppressOffers()
        Description copied from interface: SchedulerDriver
        Informs Mesos master to stop sending offers to the framework (i.e. to suppress all roles of the framework). To resume getting offers, the scheduler can call reviveOffers() or set the suppressed roles explicitly via updateFramework(). NOTE: If the framework is not connected to the master, all the roles will be added to the set of suppressed roles in the driver, and an up-to-date suppressed roles set will be sent to the master during re-registration.
        Specified by:
        suppressOffers in interface SchedulerDriver
        Returns:
        The state of the driver after the call.
        See Also:
        Protos.Status

      • suppressOffers

        public Protos.Status suppressOffers(java.util.Collection<java.lang.String> roles)
        Description copied from interface: SchedulerDriver
        Adds the roles to the suppressed set. If the framework is not connected to the master, an up-to-date set of suppressed roles will be sent to the master during re-registration.
        Specified by:
        suppressOffers in interface SchedulerDriver
        Parameters:
        roles - The collection of framework roles to be suppressed. If empty, the method does nothing.
        Returns:
        The state of the driver after the call.
        See Also:
        Protos.Status

      • acknowledgeStatusUpdate

        public Protos.Status acknowledgeStatusUpdate(Protos.TaskStatus status)
        Description copied from interface: SchedulerDriver
        Acknowledges the status update. This should only be called once the status update is processed durably by the scheduler. Not that explicit acknowledgements must be requested via the constructor argument, otherwise a call to this method will cause the driver to crash.
        Specified by:
        acknowledgeStatusUpdate in interface SchedulerDriver
        Parameters:
        status - The status to acknowledge.
        Returns:
        The state of the driver after the call.
        See Also:
        Protos.TaskStatus

      • sendFrameworkMessage

        public Protos.Status sendFrameworkMessage(Protos.ExecutorID executorId,
 Protos.SlaveID slaveId,
 byte[] data)
        Description copied from interface: SchedulerDriver
        Sends a message from the framework to one of its executors. These messages are best effort; do not expect a framework message to be retransmitted in any reliable fashion.
        Specified by:
        sendFrameworkMessage in interface SchedulerDriver
        Parameters:
        executorId - The ID of the executor to send the message to.
        slaveId - The ID of the slave that is running the executor.
        data - The message.
        Returns:
        The state of the driver after the call.
        See Also:
        Protos.ExecutorID, Protos.SlaveID

      • reconcileTasks

        public Protos.Status reconcileTasks(java.util.Collection<Protos.TaskStatus> statuses)
        Description copied from interface: SchedulerDriver
        Allows the framework to query the status for non-terminal tasks. This causes the master to send back the latest task status for each task in 'statuses', if possible. Tasks that are no longer known will result in a TASK_LOST update. If statuses is empty, then the master will send the latest status for each task currently known.
        Specified by:
        reconcileTasks in interface SchedulerDriver
        Parameters:
        statuses - The collection of non-terminal TaskStatuses to reconcile.
        Returns:
        The state of the driver after the call.
        See Also:
        Protos.TaskStatus, Protos.SlaveID

      • updateFramework

        @Deprecated
public Protos.Status updateFramework(Protos.FrameworkInfo frameworkInfo,
 java.util.Collection<java.lang.String> suppressedRoles)
        Deprecated. Replaced by #updateFramework(FrameworkInfo, Collection, OfferConstraints) NOTE: The underlying JNI method exists only to maintain compatibility of newer versions of libmesos-java.so with older versions of mesos.jar
        Specified by:
        updateFramework in interface SchedulerDriver

      • initialize

        protected void initialize()

      • finalize

        protected void finalize()
        Overrides:
        finalize in class java.lang.Object
Skip navigation links

AltStyle によって変換されたページ (->オリジナル) /